Back ArrowExplore all topics

Crawler Libraries - Page 13

n
not-your-average-web-crawlerby tijme

Python dotstar image 110 dotVersion:Currentdot
License: Permissive (MIT)

A web crawler (for bug hunting) that gathers more than you can imagine.

Support
    Quality
      Security
        License
          Reuse
            S
            SinaWeiboSpiderby wen-fei

            Python dotstar image 109 dotVersion:Currentdot
            License: Permissive (MIT)

            A web spider for Sina Weibo, based on Scrapy framework and mongodb database.

            Support
              Quality
                Security
                  License
                    Reuse
                      g
                      geetestby Kevin-Cherish

                      Python dotstar image 109 dotVersion:Currentdot
                      License: Permissive (MIT)

                      滑动验证码，希望对你们有所帮助❤️

                      Support
                        Quality
                          Security
                            License
                              Reuse
                                v
                                vipMusicby weitw

                                Python dotstar image 109 dotVersion:Currentdot
                                no licences License: No License (No License)

                                python 爬虫，下载一些vip音乐（网易云、酷狗、ＱＱ音乐）

                                Support
                                  Quality
                                    Security
                                      License
                                        Reuse
                                          b
                                          blinkist-m4a-downloaderby luckylittle

                                          Go dotstar image 109 dotVersion:Currentdot
                                          License: Permissive (MIT)

                                          Grabs all of the audio files from all of the Blinkist books

                                          Support
                                            Quality
                                              Security
                                                License
                                                  Reuse
                                                    d
                                                    django-covid19by leafcoder

                                                    Python dotstar image 108 dotVersion:Currentdot
                                                    License: Permissive (MIT)

                                                    实时接口获取中国各个城市、省份、国家的新型冠状肺炎（新冠肺炎 / 2019-nCoV / Covid-19）。疫情数据以及整体统计详情，新增美国各州统计、每日疫情数据 API。爬虫实时追踪新冠疫情变化，数据来自丁香园和 covidtracking.com。数据大屏示例：http://ncov.leafcoder.cn/demo/ 项目文档：http://ncov.leafcoder.cn/docs/

                                                    Support
                                                      Quality
                                                        Security
                                                          License
                                                            Reuse
                                                              r
                                                              redis-lockby mlanett

                                                              Ruby dotstar image 108 dotVersion:Currentdot
                                                              License: Permissive (MIT)

                                                              Pessimistic locking using Redis

                                                              Support
                                                                Quality
                                                                  Security
                                                                    License
                                                                      Reuse
                                                                        z
                                                                        zhihu-spiderby wuomzfx

                                                                        JavaScript dotstar image 108 dotVersion:Currentdot
                                                                        no licences License: No License (No License)

                                                                        知乎爬虫程序，定时跟踪问题数据，定时推送热门话题

                                                                        Support
                                                                          Quality
                                                                            Security
                                                                              License
                                                                                Reuse
                                                                                  y
                                                                                  yue-spiderby 3394772548

                                                                                  Go dotstar image 108 dotVersion:Currentdot
                                                                                  no licences License: No License (No License)

                                                                                  手机淘宝App 闲鱼App 相关爬虫 学习测试

                                                                                  Support
                                                                                    Quality
                                                                                      Security
                                                                                        License
                                                                                          Reuse
                                                                                            k
                                                                                            krawlerby brianmadden

                                                                                            Kotlin dotstar image 108 dotVersion:Currentdot
                                                                                            License: Permissive (MIT)

                                                                                            A web crawling framework written in Kotlin

                                                                                            Support
                                                                                              Quality
                                                                                                Security
                                                                                                  License
                                                                                                    Reuse
                                                                                                      s
                                                                                                      scrapy-inline-requestsby rmax

                                                                                                      Python dotstar image 107 dotVersion:Currentdot
                                                                                                      License: Permissive (MIT)

                                                                                                      A decorator to write coroutine-like spider callbacks.

                                                                                                      Support
                                                                                                        Quality
                                                                                                          Security
                                                                                                            License
                                                                                                              Reuse
                                                                                                                g
                                                                                                                go-pkg-spiderby suosi-inc

                                                                                                                Go dotstar image 107 dotVersion:Currentdot
                                                                                                                License: Permissive (Apache-2.0)

                                                                                                                一个 Golang 实现的相对智能、无需规则维护的通用新闻网站数据提取工具库。含域名探测、网页编码语种识别、网页链接分类提取、网页新闻要素抽取以及新闻正文抽取等组件。

                                                                                                                Support
                                                                                                                  Quality
                                                                                                                    Security
                                                                                                                      License
                                                                                                                        Reuse
                                                                                                                          c
                                                                                                                          crawlerby brantou

                                                                                                                          Python dotstar image 106 dotVersion:Currentdot
                                                                                                                          License: Permissive (MIT)

                                                                                                                          爬虫, http代理, 模拟登陆!

                                                                                                                          Support
                                                                                                                            Quality
                                                                                                                              Security
                                                                                                                                License
                                                                                                                                  Reuse
                                                                                                                                    w
                                                                                                                                    web_scraping_and_data_analysisby keejo125

                                                                                                                                    Python dotstar image 106 dotVersion:Currentdot
                                                                                                                                    License: Permissive (MIT)

                                                                                                                                    网络爬虫和数据分析，当当、豆瓣、知乎、猫眼、微信公众号、联想官网、今日头条爬虫

                                                                                                                                    Support
                                                                                                                                      Quality
                                                                                                                                        Security
                                                                                                                                          License
                                                                                                                                            Reuse
                                                                                                                                              p
                                                                                                                                              python-spidersby tinygeeker

                                                                                                                                              Python dotstar image 106 dotVersion:Currentdot
                                                                                                                                              License: Permissive (MIT)

                                                                                                                                              🌈 Python网络爬虫实战：王者荣耀超清壁纸、抖音无水印视频、M3U8推流视频、正方系统、财务报表、美女帅哥图片、CSDN阅读量、淘宝、京东、网易云、B站、12306、抖音、笔趣阁、漫画小说音乐电影下载等

                                                                                                                                              Support
                                                                                                                                                Quality
                                                                                                                                                  Security
                                                                                                                                                    License
                                                                                                                                                      Reuse
                                                                                                                                                        p
                                                                                                                                                        python-learningby happyjared

                                                                                                                                                        Python dotstar image 105 dotVersion:Currentdot
                                                                                                                                                        no licences License: No License (No License)

                                                                                                                                                        Those years of learning Python - 这些年学习的Python

                                                                                                                                                        Support
                                                                                                                                                          Quality
                                                                                                                                                            Security
                                                                                                                                                              License
                                                                                                                                                                Reuse
                                                                                                                                                                  D
                                                                                                                                                                  Daily_scriptsby x1ah

                                                                                                                                                                  Python dotstar image 105 dotVersion:Currentdot
                                                                                                                                                                  License: Permissive (MIT)

                                                                                                                                                                  日常小脚本，懒人欢乐多。

                                                                                                                                                                  Support
                                                                                                                                                                    Quality
                                                                                                                                                                      Security
                                                                                                                                                                        License
                                                                                                                                                                          Reuse
                                                                                                                                                                            d
                                                                                                                                                                            decoration-design-crawlerby imflyn

                                                                                                                                                                            Python dotstar image 105 dotVersion:Currentdot
                                                                                                                                                                            no licences License: No License (No License)

                                                                                                                                                                            土巴兔和谷居装修网站爬虫

                                                                                                                                                                            Support
                                                                                                                                                                              Quality
                                                                                                                                                                                Security
                                                                                                                                                                                  License
                                                                                                                                                                                    Reuse
                                                                                                                                                                                      v
                                                                                                                                                                                      video_spiderby CoderCharm

                                                                                                                                                                                      Python dotstar image 105 dotVersion:Currentdot
                                                                                                                                                                                      no licences License: No License (No License)

                                                                                                                                                                                      😛 源视频mp4链接获取: toutiao今日头条app视频;🍉xigua西瓜视频; 🐧tencent腾讯视频; 🎼douyin抖音分享短链接解析，获取无水印播放链接;

                                                                                                                                                                                      Support
                                                                                                                                                                                        Quality
                                                                                                                                                                                          Security
                                                                                                                                                                                            License
                                                                                                                                                                                              Reuse
                                                                                                                                                                                                b
                                                                                                                                                                                                bing-image-spiderby chengquan223

                                                                                                                                                                                                C# dotstar image 105 dotVersion:Currentdot
                                                                                                                                                                                                no licences License: No License (No License)

                                                                                                                                                                                                bing必应官网每日背景图片爬取

                                                                                                                                                                                                Support
                                                                                                                                                                                                  Quality
                                                                                                                                                                                                    Security
                                                                                                                                                                                                      License
                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                          D
                                                                                                                                                                                                          DotnetCrawlerby mehmetozkaya

                                                                                                                                                                                                          C# dotstar image 105 dotVersion:Currentdot
                                                                                                                                                                                                          no licences License: No License (No License)

                                                                                                                                                                                                          DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

                                                                                                                                                                                                          Support
                                                                                                                                                                                                            Quality
                                                                                                                                                                                                              Security
                                                                                                                                                                                                                License
                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                    a
                                                                                                                                                                                                                    andvaranautby glouw

                                                                                                                                                                                                                    C dotstar image 105 dotVersion:Currentdot
                                                                                                                                                                                                                    License: Strong Copyleft (GPL-3.0)

                                                                                                                                                                                                                    A dungeon crawler

                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                          License
                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                              z
                                                                                                                                                                                                                              zhihu-crawlerby cpselvis

                                                                                                                                                                                                                              Python dotstar image 104 dotVersion:Currentdot
                                                                                                                                                                                                                              License: Permissive (MIT)

                                                                                                                                                                                                                              A zhihu user information crawler, which will collect some useful message including username, education, profession, follower and folling count.

                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                        s
                                                                                                                                                                                                                                        scrapy-puppeteerby clemfromspace

                                                                                                                                                                                                                                        Python dotstar image 104 dotVersion:Currentdot
                                                                                                                                                                                                                                        License: Permissive (MIT)

                                                                                                                                                                                                                                        Scrapy + Puppeteer

                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                  A
                                                                                                                                                                                                                                                  AmazonRobotby WuLC

                                                                                                                                                                                                                                                  Python dotstar image 104 dotVersion:Currentdot
                                                                                                                                                                                                                                                  License: Permissive (MIT)

                                                                                                                                                                                                                                                  Amazon商品引流的 python 爬虫

                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                            s
                                                                                                                                                                                                                                                            scriptsby jhaddix

                                                                                                                                                                                                                                                            Python dotstar image 104 dotVersion:Currentdot
                                                                                                                                                                                                                                                            no licences License: No License (No License)

                                                                                                                                                                                                                                                            Usefull stuff from around teh internetz

                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                      s
                                                                                                                                                                                                                                                                      ss-angularby polidore

                                                                                                                                                                                                                                                                      JavaScript dotstar image 104 dotVersion:Currentdot
                                                                                                                                                                                                                                                                      no licences License: No License (No License)

                                                                                                                                                                                                                                                                      AngularJS integrations to SocketStream

                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                L
                                                                                                                                                                                                                                                                                LinkCrawlerby hmol

                                                                                                                                                                                                                                                                                C# dotstar image 104 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                License: Permissive (MIT)

                                                                                                                                                                                                                                                                                Find broken links in webpage

                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                          g
                                                                                                                                                                                                                                                                                          go-htmltableby nfx

                                                                                                                                                                                                                                                                                          Go dotstar image 104 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                          License: Permissive (MIT)

                                                                                                                                                                                                                                                                                          Structured HTML table data extraction from URLs in Go that has almost no external dependencies

                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                    s
                                                                                                                                                                                                                                                                                                    scrapy-spidersby dcondrey

                                                                                                                                                                                                                                                                                                    Python dotstar image 103 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                    License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                    Collection of python scripts I have created to crawl various websites, mostly for lead generation projects to match keywords and collect email addresses and post URLs

                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                              C
                                                                                                                                                                                                                                                                                                              CrawlerPackby abola

                                                                                                                                                                                                                                                                                                              Java dotstar image 103 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                              License: Permissive (Apache-2.0)

                                                                                                                                                                                                                                                                                                              Java 網路資料爬蟲包

                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                        P
                                                                                                                                                                                                                                                                                                                        PassiveScannerby jjf012

                                                                                                                                                                                                                                                                                                                        Python dotstar image 103 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                        License: Strong Copyleft (GPL-3.0)

                                                                                                                                                                                                                                                                                                                        a passive scanner based on Mitmproxy and Arachni

                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                  s
                                                                                                                                                                                                                                                                                                                                  spidermonkeyby tlhunter

                                                                                                                                                                                                                                                                                                                                  PHP dotstar image 103 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                  no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                  DEFUNCT: PHP Web Spider from 2011

                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                            s
                                                                                                                                                                                                                                                                                                                                            spider_news_allby hailong0707-zz

                                                                                                                                                                                                                                                                                                                                            Python dotstar image 102 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                            no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                            Scrapy Spider for 各种新闻网站

                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                      d
                                                                                                                                                                                                                                                                                                                                                      demiurgeby matiasb

                                                                                                                                                                                                                                                                                                                                                      Python dotstar image 102 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                      License: Permissive (BSD-3-Clause)

                                                                                                                                                                                                                                                                                                                                                      PyQuery-based scraping micro-framework.

                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                M
                                                                                                                                                                                                                                                                                                                                                                Mtimeby dongweiming

                                                                                                                                                                                                                                                                                                                                                                Python dotstar image 102 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                                                A spider... ^.^

                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                          p
                                                                                                                                                                                                                                                                                                                                                                          pspiderby zhao94254

                                                                                                                                                                                                                                                                                                                                                                          Python dotstar image 102 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                          License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                                                                          一个简单的分布式爬虫框架

                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                    s
                                                                                                                                                                                                                                                                                                                                                                                    scrapy-monitorby ioiogoo

                                                                                                                                                                                                                                                                                                                                                                                    Python dotstar image 102 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                    License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                                                                                    scrapy-monitor，实现爬虫可视化，监控实时状态

                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                              a
                                                                                                                                                                                                                                                                                                                                                                                              antispiderby dytttf

                                                                                                                                                                                                                                                                                                                                                                                              JavaScript dotstar image 101 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                              no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                        b
                                                                                                                                                                                                                                                                                                                                                                                                        bullshit-job-titlesby bullgit

                                                                                                                                                                                                                                                                                                                                                                                                        HTML dotstar image 101 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                        no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                                                                                        the shittiest job titles you'll ever see

                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                  I
                                                                                                                                                                                                                                                                                                                                                                                                                  Investigoby tdh8316

                                                                                                                                                                                                                                                                                                                                                                                                                  Go dotstar image 101 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                  License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                                                                                                                  🔎 Find usernames and download their data across social media.

                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                            k
                                                                                                                                                                                                                                                                                                                                                                                                                            kezplez-nxby tesnos

                                                                                                                                                                                                                                                                                                                                                                                                                            C dotstar image 101 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                            License: Proprietary (Proprietary)

                                                                                                                                                                                                                                                                                                                                                                                                                            The "easiest" way to get all 70+ Nintendo Switch keys to use with hactool!

                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                      z
                                                                                                                                                                                                                                                                                                                                                                                                                                      zyte-smartproxy-headless-proxyby zytedata

                                                                                                                                                                                                                                                                                                                                                                                                                                      Go dotstar image 101 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                      License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                                                                                                                                      A complimentary proxy to help to use SPM with headless browsers

                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                Z
                                                                                                                                                                                                                                                                                                                                                                                                                                                Zhihu_Crawlerby salamer

                                                                                                                                                                                                                                                                                                                                                                                                                                                Python dotstar image 100 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                License: Permissive (Apache-2.0)

                                                                                                                                                                                                                                                                                                                                                                                                                                                a crawler for zhihu

                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                          d
                                                                                                                                                                                                                                                                                                                                                                                                                                                          dl_courseraby feng-lei

                                                                                                                                                                                                                                                                                                                                                                                                                                                          Python dotstar image 100 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                          License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                                                                                                                                                          A simple, fast, and reliable Coursera crawling & downloading tool

                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                    C
                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Crawlicby Ganapati

                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Python dotstar image 100 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License: Proprietary (Proprietary)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Web recon tool (find temporary files, parse robots.txt, search some folders, google dorks and search domains hosted on same server)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                              p
                                                                                                                                                                                                                                                                                                                                                                                                                                                                              platform_agentby basecamp

                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Ruby dotstar image 100 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Parse user agent to deduce the platform

                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        b
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        beian_miit_spiderby Ithrael

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Python dotstar image 99 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License: Strong Copyleft (GPL-2.0)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        一个工业和信息化部ICP备案查询的爬虫

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  z
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  ziroom_realtime_spiderby facert

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Python dotstar image 99 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  自如实时房源提醒

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            c
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            capturerby Litreily

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Python dotstar image 99 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            capture pictures from website like sina, lofter, huaban and so on

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      n
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      not-your-average-web-crawlerby tijme
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      A web crawler (for bug hunting) that gathers more than you can imagine.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Pythondotstar image 110dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                S
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                SinaWeiboSpiderby wen-fei
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                A web spider for Sina Weibo, based on Scrapy framework and mongodb database.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Pythondotstar image 109dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          g
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          geetestby Kevin-Cherish
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          滑动验证码，希望对你们有所帮助❤️
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Pythondotstar image 109dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    v
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    vipMusicby weitw
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    python 爬虫，下载一些vip音乐（网易云、酷狗、ＱＱ音乐）
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Pythondotstar image 109dotUpdated: 2 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              b
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              blinkist-m4a-downloaderby luckylittle
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Grabs all of the audio files from all of the Blinkist books
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Godotstar image 109dotUpdated: 2 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        d
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        django-covid19by leafcoder
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        实时接口获取中国各个城市、省份、国家的新型冠状肺炎（新冠肺炎 / 2019-nCoV / Covid-19）。疫情数据以及整体统计详情，新增美国各州统计、每日疫情数据 API。爬虫实时追踪新冠疫情变化，数据来自丁香园和 covidtracking.com。数据大屏示例：http://ncov.leafcoder.cn/demo/ 项目文档：http://ncov.leafcoder.cn/docs/
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Pythondotstar image 108dotUpdated: 3 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  r
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  redis-lockby mlanett
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Pessimistic locking using Redis
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Rubydotstar image 108dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            z
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            zhihu-spiderby wuomzfx
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            知乎爬虫程序，定时跟踪问题数据，定时推送热门话题
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            JavaScriptdotstar image 108dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      y
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      yue-spiderby 3394772548
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      手机淘宝App 闲鱼App 相关爬虫 学习测试
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Godotstar image 108dotUpdated: 3 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                k
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                krawlerby brianmadden
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                A web crawling framework written in Kotlin
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Kotlindotstar image 108dotUpdated: 3 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          s
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          scrapy-inline-requestsby rmax
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          A decorator to write coroutine-like spider callbacks.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Pythondotstar image 107dotUpdated: 2 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    g
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    go-pkg-spiderby suosi-inc
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    一个 Golang 实现的相对智能、无需规则维护的通用新闻网站数据提取工具库。含域名探测、网页编码语种识别、网页链接分类提取、网页新闻要素抽取以及新闻正文抽取等组件。
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Godotstar image 107dotUpdated: 1 y agodot License: Permissive (Apache-2.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              c
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              crawlerby brantou
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              爬虫, http代理, 模拟登陆!
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Pythondotstar image 106dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        w
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        web_scraping_and_data_analysisby keejo125
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        网络爬虫和数据分析，当当、豆瓣、知乎、猫眼、微信公众号、联想官网、今日头条爬虫
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Pythondotstar image 106dotUpdated: 3 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  p
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  python-spidersby tinygeeker
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  🌈 Python网络爬虫实战：王者荣耀超清壁纸、抖音无水印视频、M3U8推流视频、正方系统、财务报表、美女帅哥图片、CSDN阅读量、淘宝、京东、网易云、B站、12306、抖音、笔趣阁、漫画小说音乐电影下载等
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Pythondotstar image 106dotUpdated: 2 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            p
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            python-learningby happyjared
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Those years of learning Python - 这些年学习的Python
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Pythondotstar image 105dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      D
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Daily_scriptsby x1ah
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      日常小脚本，懒人欢乐多。
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Pythondotstar image 105dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                d
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                decoration-design-crawlerby imflyn
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                土巴兔和谷居装修网站爬虫
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Pythondotstar image 105dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          v
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          video_spiderby CoderCharm
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          😛 源视频mp4链接获取: toutiao今日头条app视频;🍉xigua西瓜视频; 🐧tencent腾讯视频; 🎼douyin抖音分享短链接解析，获取无水印播放链接;
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Pythondotstar image 105dotUpdated: 3 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    b
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    bing-image-spiderby chengquan223
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    bing必应官网每日背景图片爬取
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    C#dotstar image 105dotUpdated: 5 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              D
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              DotnetCrawlerby mehmetozkaya
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              C#dotstar image 105dotUpdated: 3 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        a
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        andvaranautby glouw
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        A dungeon crawler
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Cdotstar image 105dotUpdated: 3 y agodot License: Strong Copyleft (GPL-3.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  z
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  zhihu-crawlerby cpselvis
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  A zhihu user information crawler, which will collect some useful message including username, education, profession, follower and folling count.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Pythondotstar image 104dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            s
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            scrapy-puppeteerby clemfromspace
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Scrapy + Puppeteer
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Pythondotstar image 104dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      A
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      AmazonRobotby WuLC
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Amazon商品引流的 python 爬虫
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Pythondotstar image 104dotUpdated: 3 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                s
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                scriptsby jhaddix
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Usefull stuff from around teh internetz
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Pythondotstar image 104dotUpdated: 2 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          s
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          ss-angularby polidore
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          AngularJS integrations to SocketStream
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          JavaScriptdotstar image 104dotUpdated: 5 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    L
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    LinkCrawlerby hmol
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Find broken links in webpage
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    C#dotstar image 104dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              g
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              go-htmltableby nfx
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Structured HTML table data extraction from URLs in Go that has almost no external dependencies
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Godotstar image 104dotUpdated: 2 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        s
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        scrapy-spidersby dcondrey
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Collection of python scripts I have created to crawl various websites, mostly for lead generation projects to match keywords and collect email addresses and post URLs
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Pythondotstar image 103dotUpdated: 3 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  C
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  CrawlerPackby abola
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Java 網路資料爬蟲包
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Javadotstar image 103dotUpdated: 4 y agodot License: Permissive (Apache-2.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            P
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            PassiveScannerby jjf012
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            a passive scanner based on Mitmproxy and Arachni
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Pythondotstar image 103dotUpdated: 4 y agodot License: Strong Copyleft (GPL-3.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      s
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      spidermonkeyby tlhunter
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      DEFUNCT: PHP Web Spider from 2011
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      PHPdotstar image 103dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                s
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                spider_news_allby hailong0707-zz
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Scrapy Spider for 各种新闻网站
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Pythondotstar image 102dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          d
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          demiurgeby matiasb
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          PyQuery-based scraping micro-framework.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Pythondotstar image 102dotUpdated: 4 y agodot License: Permissive (BSD-3-Clause)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    M
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Mtimeby dongweiming
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    A spider... ^.^
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Pythondotstar image 102dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              p
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              pspiderby zhao94254
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              一个简单的分布式爬虫框架
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Pythondotstar image 102dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        s
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        scrapy-monitorby ioiogoo
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        scrapy-monitor，实现爬虫可视化，监控实时状态
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Pythondotstar image 102dotUpdated: 3 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  a
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  antispiderby dytttf
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  JavaScriptdotstar image 101dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            b
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            bullshit-job-titlesby bullgit
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            the shittiest job titles you'll ever see
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            HTMLdotstar image 101dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      I
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Investigoby tdh8316
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      🔎 Find usernames and download their data across social media.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Godotstar image 101dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                k
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                kezplez-nxby tesnos
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                The "easiest" way to get all 70+ Nintendo Switch keys to use with hactool!
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Cdotstar image 101dotUpdated: 4 y agodot License: Proprietary (Proprietary)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          z
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          zyte-smartproxy-headless-proxyby zytedata
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          A complimentary proxy to help to use SPM with headless browsers
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Godotstar image 101dotUpdated: 2 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Z
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Zhihu_Crawlerby salamer
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    a crawler for zhihu
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Pythondotstar image 100dotUpdated: 4 y agodot License: Permissive (Apache-2.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              d
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              dl_courseraby feng-lei
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              A simple, fast, and reliable Coursera crawling & downloading tool
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Pythondotstar image 100dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        C
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Crawlicby Ganapati
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Web recon tool (find temporary files, parse robots.txt, search some folders, google dorks and search domains hosted on same server)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Pythondotstar image 100dotUpdated: 4 y agodot License: Proprietary (Proprietary)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  p
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  platform_agentby basecamp
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Parse user agent to deduce the platform
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Rubydotstar image 100dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            b
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            beian_miit_spiderby Ithrael
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            一个工业和信息化部ICP备案查询的爬虫
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Pythondotstar image 99dotUpdated: 1 y agodot License: Strong Copyleft (GPL-2.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      z
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      ziroom_realtime_spiderby facert
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      自如实时房源提醒
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Pythondotstar image 99dotUpdated: 2 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                c
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                capturerby Litreily
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                capture pictures from website like sina, lofter, huaban and so on
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Pythondotstar image 99dotUpdated: 2 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Open Weaver – Develop Applications Faster with Open Source

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          kandi

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Community and Support

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Company

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Follow

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          • © 2023 Open Weaver Inc.