Back ArrowExplore all topics

Crawler Libraries - Page 19

c
crawdadby schollz

Go dotstar image 57 dotVersion:Currentdot
License: Permissive (MIT)

Cross-platform persistent and distributed web crawler :crab:

Support
    Quality
      Security
        License
          Reuse
            j
            js_blockby webcoding

            HTML dotstar image 57 dotVersion:Currentdot
            License: Permissive (MIT)

            研究学习各种拦截：反爬虫、拦截ad、防广告注入、斗黄牛等

            Support
              Quality
                Security
                  License
                    Reuse
                      C
                      ComicSpiderby QuantumLiu

                      Python dotstar image 56 dotVersion:Currentdot
                      License: Strong Copyleft (GPL-3.0)

                      动漫之家漫画站电脑版原图爬虫

                      Support
                        Quality
                          Security
                            License
                              Reuse
                                p
                                parseNTFSby Heurs

                                Python dotstar image 56 dotVersion:Currentdot
                                no licences License: No License (No License)

                                Simple NTFS crawler.

                                Support
                                  Quality
                                    Security
                                      License
                                        Reuse
                                          c
                                          cyworld-botby leegeunhyeok

                                          Python dotstar image 56 dotVersion:Currentdot
                                          License: Permissive (MIT)

                                          🤖 Cyworld image crawler

                                          Support
                                            Quality
                                              Security
                                                License
                                                  Reuse
                                                    E
                                                    Email-Crawler-Lead-Generatorby amitupreti

                                                    Python dotstar image 56 dotVersion:Currentdot
                                                    License: Permissive (MIT)

                                                    This email crawler will visit all pages of a provided website and parse and save emails found to a csv file.

                                                    Support
                                                      Quality
                                                        Security
                                                          License
                                                            Reuse
                                                              S
                                                              Scrapy-Craigslistby GoTrained

                                                              Python dotstar image 56 dotVersion:Currentdot
                                                              no licences License: No License (No License)

                                                              Web Scraping Craigslist's Engineering Jobs in NY with Scrapy

                                                              Support
                                                                Quality
                                                                  Security
                                                                    License
                                                                      Reuse
                                                                        t
                                                                        talospiderby howie6879

                                                                        Python dotstar image 56 dotVersion:Currentdot
                                                                        no licences License: No License (No License)

                                                                        talospider - A simple,lightweight scraping micro-framework

                                                                        Support
                                                                          Quality
                                                                            Security
                                                                              License
                                                                                Reuse
                                                                                  e
                                                                                  echarts3-docsby gogo1217

                                                                                  JavaScript dotstar image 56 dotVersion:Currentdot
                                                                                  no licences License: No License (No License)

                                                                                  echarts3 文档、实例，http://echarts.baidu.com/ 离线版

                                                                                  Support
                                                                                    Quality
                                                                                      Security
                                                                                        License
                                                                                          Reuse
                                                                                            M
                                                                                            MumiGivePby wabilin

                                                                                            JavaScript dotstar image 56 dotVersion:Currentdot
                                                                                            License: Weak Copyleft (LGPL-3.0)

                                                                                            Give money to others automatically on PTT(Taiwan BBS Site)

                                                                                            Support
                                                                                              Quality
                                                                                                Security
                                                                                                  License
                                                                                                    Reuse
                                                                                                      J
                                                                                                      Java-Carwler-Technologyby soberqian

                                                                                                      Java dotstar image 55 dotVersion:Currentdot
                                                                                                      no licences License: No License (No License)

                                                                                                      网络数据采集技术—Java网络爬虫 (书稿完整代码，涉及网络爬虫的各种技术和知识点)

                                                                                                      Support
                                                                                                        Quality
                                                                                                          Security
                                                                                                            License
                                                                                                              Reuse
                                                                                                                s
                                                                                                                second-spiderby kenshinx

                                                                                                                Python dotstar image 55 dotVersion:Currentdot
                                                                                                                no licences License: No License (No License)

                                                                                                                one more spider based on gevent requests pyquery

                                                                                                                Support
                                                                                                                  Quality
                                                                                                                    Security
                                                                                                                      License
                                                                                                                        Reuse
                                                                                                                          M
                                                                                                                          MyCrawlerby netcan

                                                                                                                          Python dotstar image 55 dotVersion:Currentdot
                                                                                                                          License: Strong Copyleft (GPL-3.0)

                                                                                                                          我的爬虫合集

                                                                                                                          Support
                                                                                                                            Quality
                                                                                                                              Security
                                                                                                                                License
                                                                                                                                  Reuse
                                                                                                                                    S
                                                                                                                                    SpiderDemoby SFLAQiu

                                                                                                                                    Python dotstar image 55 dotVersion:Currentdot
                                                                                                                                    no licences License: No License (No License)

                                                                                                                                    :trollface: 爬虫Demo，基于Python实现

                                                                                                                                    Support
                                                                                                                                      Quality
                                                                                                                                        Security
                                                                                                                                          License
                                                                                                                                            Reuse
                                                                                                                                              f
                                                                                                                                              facebotby pun1sh3r

                                                                                                                                              Python dotstar image 55 dotVersion:Currentdot
                                                                                                                                              no licences License: No License (No License)

                                                                                                                                              A facebook profile and reconnaissance system

                                                                                                                                              Support
                                                                                                                                                Quality
                                                                                                                                                  Security
                                                                                                                                                    License
                                                                                                                                                      Reuse
                                                                                                                                                        W
                                                                                                                                                        WebCollector-Pythonby CrawlScript

                                                                                                                                                        Python dotstar image 55 dotVersion:Currentdot
                                                                                                                                                        License: Strong Copyleft (GPL-3.0)

                                                                                                                                                        WebCollector-Python is an open source web crawler framework based on Python.It provides some simple interfaces for crawling the Web,you can setup a multi-threaded web crawler in less than 5 minutes.

                                                                                                                                                        Support
                                                                                                                                                          Quality
                                                                                                                                                            Security
                                                                                                                                                              License
                                                                                                                                                                Reuse
                                                                                                                                                                  s
                                                                                                                                                                  spiderMHCBGby tangwei123

                                                                                                                                                                  Python dotstar image 55 dotVersion:Currentdot
                                                                                                                                                                  no licences License: No License (No License)

                                                                                                                                                                  python scrapy 扒取梦幻藏宝阁的上架的账号信息，然后分析出符合我要求的账号，发送带账号网址的链接地址到我的邮箱

                                                                                                                                                                  Support
                                                                                                                                                                    Quality
                                                                                                                                                                      Security
                                                                                                                                                                        License
                                                                                                                                                                          Reuse
                                                                                                                                                                            t
                                                                                                                                                                            torrents-crawlerby yasoob

                                                                                                                                                                            Python dotstar image 55 dotVersion:Currentdot
                                                                                                                                                                            License: Weak Copyleft (MPL-2.0)

                                                                                                                                                                            This is a scrapy project in which I have implemented several crawlers for different torrent and direct link websites.

                                                                                                                                                                            Support
                                                                                                                                                                              Quality
                                                                                                                                                                                Security
                                                                                                                                                                                  License
                                                                                                                                                                                    Reuse
                                                                                                                                                                                      a
                                                                                                                                                                                      autohome_crawlerby William-Sang

                                                                                                                                                                                      Python dotstar image 55 dotVersion:Currentdot
                                                                                                                                                                                      no licences License: No License (No License)

                                                                                                                                                                                      使用Python requests 和 BeautifulSoup 开发爬虫。 抓取汽车之家中，汽车的基本信息（车型，品牌，报价等）

                                                                                                                                                                                      Support
                                                                                                                                                                                        Quality
                                                                                                                                                                                          Security
                                                                                                                                                                                            License
                                                                                                                                                                                              Reuse
                                                                                                                                                                                                P
                                                                                                                                                                                                PicCrawlerby fengzhizi715

                                                                                                                                                                                                Java dotstar image 55 dotVersion:Currentdot
                                                                                                                                                                                                License: Permissive (Apache-2.0)

                                                                                                                                                                                                使用RxJava2 和 Java 8的特性开发的图片爬虫

                                                                                                                                                                                                Support
                                                                                                                                                                                                  Quality
                                                                                                                                                                                                    Security
                                                                                                                                                                                                      License
                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                          s
                                                                                                                                                                                                          scrapy_modelby rochacbruno-archive

                                                                                                                                                                                                          Python dotstar image 55 dotVersion:Currentdot
                                                                                                                                                                                                          License: Permissive (BSD-3-Clause)

                                                                                                                                                                                                          A helper to create web scrapers using scrapy selector in a Model based structure

                                                                                                                                                                                                          Support
                                                                                                                                                                                                            Quality
                                                                                                                                                                                                              Security
                                                                                                                                                                                                                License
                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                    d
                                                                                                                                                                                                                    dgeni-angularby petebacondarwin

                                                                                                                                                                                                                    JavaScript dotstar image 55 dotVersion:Currentdot
                                                                                                                                                                                                                    License: Permissive (Apache-2.0)

                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                          License
                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                              m
                                                                                                                                                                                                                              movie-getterby jinzhongjia

                                                                                                                                                                                                                              Go dotstar image 55 dotVersion:Currentdot
                                                                                                                                                                                                                              License: Permissive (MIT)

                                                                                                                                                                                                                              Go编写的影视资源采集器

                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                        m
                                                                                                                                                                                                                                        metacritic_apiby melroy89

                                                                                                                                                                                                                                        PHP dotstar image 55 dotVersion:Currentdot
                                                                                                                                                                                                                                        License: Permissive (Apache-2.0)

                                                                                                                                                                                                                                        PHP Metacritic API - Mirror from my GitLab

                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                  L
                                                                                                                                                                                                                                                  Last-Statement-of-Death-Rowby wansho

                                                                                                                                                                                                                                                  Python dotstar image 54 dotVersion:Currentdot
                                                                                                                                                                                                                                                  License: Permissive (MIT)

                                                                                                                                                                                                                                                  Last-Statement-of-Death-Row, 人之将死，其言也善

                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                            j
                                                                                                                                                                                                                                                            just-a-spiderby Jiezhi

                                                                                                                                                                                                                                                            Python dotstar image 54 dotVersion:Currentdot
                                                                                                                                                                                                                                                            no licences License: No License (No License)

                                                                                                                                                                                                                                                            🕸Just let spider save your time.

                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                      D
                                                                                                                                                                                                                                                                      Daily-codeby rui7157

                                                                                                                                                                                                                                                                      Python dotstar image 54 dotVersion:Currentdot
                                                                                                                                                                                                                                                                      License: Strong Copyleft (GPL-3.0)

                                                                                                                                                                                                                                                                      日常代码爬虫、gui小工具等

                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                s
                                                                                                                                                                                                                                                                                simple_bank_koreaby Beomi

                                                                                                                                                                                                                                                                                Python dotstar image 54 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                License: Permissive (MIT)

                                                                                                                                                                                                                                                                                simple crawler for Korean banks with Transactions

                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                          T
                                                                                                                                                                                                                                                                                          TrustlookWannaCryToolkitby apkjet

                                                                                                                                                                                                                                                                                          Python dotstar image 54 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                          no licences License: No License (No License)

                                                                                                                                                                                                                                                                                          WannaCryToolkit scanner and removal toolkit

                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                    s
                                                                                                                                                                                                                                                                                                    spiderby binglansky

                                                                                                                                                                                                                                                                                                    Python dotstar image 54 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                    no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                    一些小爬虫 : )

                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                              c
                                                                                                                                                                                                                                                                                                              corpreditby LongYosef

                                                                                                                                                                                                                                                                                                              Python dotstar image 54 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                              no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                              国家企业信用信息官网爬虫，未获取全部企业信息，重点在设计反爬思路

                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                        c
                                                                                                                                                                                                                                                                                                                        crawlersby teampopong

                                                                                                                                                                                                                                                                                                                        Python dotstar image 54 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                        License: Strong Copyleft (AGPL-3.0)

                                                                                                                                                                                                                                                                                                                        Some quick 'n dirty web crawlers.

                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                  b
                                                                                                                                                                                                                                                                                                                                  browsercrawlerby spullara

                                                                                                                                                                                                                                                                                                                                  JavaScript dotstar image 54 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                  no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                  Crawl websites from your browser and save them in S3

                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                            w
                                                                                                                                                                                                                                                                                                                                            workerman-filemonitorby walkor

                                                                                                                                                                                                                                                                                                                                            PHP dotstar image 54 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                            no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                            监控文件更新并自动reload workerman

                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                      c
                                                                                                                                                                                                                                                                                                                                                      color-extractorby marekm4

                                                                                                                                                                                                                                                                                                                                                      Go dotstar image 54 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                      License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                                                      Simple image color extractor written in Go with no external dependencies

                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                n
                                                                                                                                                                                                                                                                                                                                                                nmaptableby jgamblin

                                                                                                                                                                                                                                                                                                                                                                HTML dotstar image 54 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                                                                Transform NMap Scans to an D3.js HTML Table

                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                          t
                                                                                                                                                                                                                                                                                                                                                                          tool-ginby bajins

                                                                                                                                                                                                                                                                                                                                                                          Go dotstar image 54 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                          no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                                                          基于go-gin框架建立减少冗余动作项目，如：下载一些工具

                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                    c
                                                                                                                                                                                                                                                                                                                                                                                    crawlBaiduWenkuby vict-cn

                                                                                                                                                                                                                                                                                                                                                                                    Python dotstar image 53 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                    License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                                                                                    这可能是爬百度文库最全的项目了

                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                              W
                                                                                                                                                                                                                                                                                                                                                                                              WebCrawlerby Misterhex

                                                                                                                                                                                                                                                                                                                                                                                              C# dotstar image 53 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                              no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                                                                              Just a simple web crawler which return crawled links as IObservable using reactive extension and async await.

                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                        c
                                                                                                                                                                                                                                                                                                                                                                                                        crawlzoneby crawlzone

                                                                                                                                                                                                                                                                                                                                                                                                        PHP dotstar image 53 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                        License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                                                                                                        Crawlzone is a fast asynchronous internet crawling framework for PHP.

                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                  b
                                                                                                                                                                                                                                                                                                                                                                                                                  box-spiderby Laichj

                                                                                                                                                                                                                                                                                                                                                                                                                  Java dotstar image 53 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                  License: Permissive (Apache-2.0)

                                                                                                                                                                                                                                                                                                                                                                                                                  SpringBoot快速开发的爬虫项目，轻易爬取A股5000+支股票250日行情百万数据，也支持腾讯新闻、凤凰资讯数据爬取

                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                            b
                                                                                                                                                                                                                                                                                                                                                                                                                            btletby neoql

                                                                                                                                                                                                                                                                                                                                                                                                                            Go dotstar image 53 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                            License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                                                                                                                            Some toolkits implements part of BT Protocol, like DHT spider.

                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                      g
                                                                                                                                                                                                                                                                                                                                                                                                                                      go-mysql-syncerby hacktmz

                                                                                                                                                                                                                                                                                                                                                                                                                                      Go dotstar image 53 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                      no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                                                                                                                      基于go-mysql & gorail二次开发的binlog解析工具，使用MQ跨机房同步，增加了修改表结构后依然能正常工作的特性，多个mysql同时处理的功能

                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                I
                                                                                                                                                                                                                                                                                                                                                                                                                                                ImageCrawlby dxsooo

                                                                                                                                                                                                                                                                                                                                                                                                                                                Python dotstar image 52 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                                                                                                                                Web Image Crawler by scrapy

                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                          d
                                                                                                                                                                                                                                                                                                                                                                                                                                                          doubanMovieCrawlerby luckterry7

                                                                                                                                                                                                                                                                                                                                                                                                                                                          Java dotstar image 52 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                          no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                                                                                                                                          doubanMovieCrawler,for collecting lastest movie

                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                    n
                                                                                                                                                                                                                                                                                                                                                                                                                                                                    news_spiderby jfzhang95

                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Python dotstar image 52 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License: Permissive (Apache-2.0)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                    新闻爬虫 (腾讯,网易,新浪,今日头条,搜狐,凤凰网,腾讯滚动新闻)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                              p
                                                                                                                                                                                                                                                                                                                                                                                                                                                                              proxycrawl-pythonby proxycrawl

                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Python dotstar image 52 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License: Permissive (Apache-2.0)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                              ProxyCrawl Python library for scraping and crawling

                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        n
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        netcrawlby ytti

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Ruby dotstar image 52 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        LLDP/CDP crawler

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  S
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  SkyScraperby JonCanning

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  C# dotstar image 52 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  An asynchronous web scraper / web crawler using async / await and Reactive Extensions

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            u
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            url-crawlerby pop-os

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Rust dotstar image 52 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Rust crate for configurable parallel web crawling, designed to crawl for content

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      c
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      crawdadby schollz
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Cross-platform persistent and distributed web crawler :crab:
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Godotstar image 57dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                j
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                js_blockby webcoding
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                研究学习各种拦截：反爬虫、拦截ad、防广告注入、斗黄牛等
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                HTMLdotstar image 57dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          C
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          ComicSpiderby QuantumLiu
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          动漫之家漫画站电脑版原图爬虫
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Pythondotstar image 56dotUpdated: 4 y agodot License: Strong Copyleft (GPL-3.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    p
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    parseNTFSby Heurs
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Simple NTFS crawler.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Pythondotstar image 56dotUpdated: 2 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              c
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              cyworld-botby leegeunhyeok
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              🤖 Cyworld image crawler
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Pythondotstar image 56dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        E
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Email-Crawler-Lead-Generatorby amitupreti
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        This email crawler will visit all pages of a provided website and parse and save emails found to a csv file.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Pythondotstar image 56dotUpdated: 2 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  S
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Scrapy-Craigslistby GoTrained
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Web Scraping Craigslist's Engineering Jobs in NY with Scrapy
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Pythondotstar image 56dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            t
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            talospiderby howie6879
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            talospider - A simple,lightweight scraping micro-framework
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Pythondotstar image 56dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      e
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      echarts3-docsby gogo1217
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      echarts3 文档、实例，http://echarts.baidu.com/ 离线版
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      JavaScriptdotstar image 56dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                M
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                MumiGivePby wabilin
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Give money to others automatically on PTT(Taiwan BBS Site)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                JavaScriptdotstar image 56dotUpdated: 3 y agodot License: Weak Copyleft (LGPL-3.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          J
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Java-Carwler-Technologyby soberqian
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          网络数据采集技术—Java网络爬虫 (书稿完整代码，涉及网络爬虫的各种技术和知识点)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Javadotstar image 55dotUpdated: 3 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    s
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    second-spiderby kenshinx
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    one more spider based on gevent requests pyquery
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Pythondotstar image 55dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              M
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              MyCrawlerby netcan
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              我的爬虫合集
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Pythondotstar image 55dotUpdated: 4 y agodot License: Strong Copyleft (GPL-3.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        S
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        SpiderDemoby SFLAQiu
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        :trollface: 爬虫Demo，基于Python实现
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Pythondotstar image 55dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  f
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  facebotby pun1sh3r
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  A facebook profile and reconnaissance system
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Pythondotstar image 55dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            W
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            WebCollector-Pythonby CrawlScript
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            WebCollector-Python is an open source web crawler framework based on Python.It provides some simple interfaces for crawling the Web,you can setup a multi-threaded web crawler in less than 5 minutes.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Pythondotstar image 55dotUpdated: 2 y agodot License: Strong Copyleft (GPL-3.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      s
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      spiderMHCBGby tangwei123
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      python scrapy 扒取梦幻藏宝阁的上架的账号信息，然后分析出符合我要求的账号，发送带账号网址的链接地址到我的邮箱
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Pythondotstar image 55dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                t
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                torrents-crawlerby yasoob
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                This is a scrapy project in which I have implemented several crawlers for different torrent and direct link websites.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Pythondotstar image 55dotUpdated: 4 y agodot License: Weak Copyleft (MPL-2.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          a
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          autohome_crawlerby William-Sang
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          使用Python requests 和 BeautifulSoup 开发爬虫。 抓取汽车之家中，汽车的基本信息（车型，品牌，报价等）
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Pythondotstar image 55dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    P
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    PicCrawlerby fengzhizi715
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    使用RxJava2 和 Java 8的特性开发的图片爬虫
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Javadotstar image 55dotUpdated: 4 y agodot License: Permissive (Apache-2.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              s
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              scrapy_modelby rochacbruno-archive
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              A helper to create web scrapers using scrapy selector in a Model based structure
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Pythondotstar image 55dotUpdated: 3 y agodot License: Permissive (BSD-3-Clause)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        d
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        dgeni-angularby petebacondarwin
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        JavaScriptdotstar image 55dotUpdated: 4 y agodot License: Permissive (Apache-2.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  m
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  movie-getterby jinzhongjia
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Go编写的影视资源采集器
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Godotstar image 55dotUpdated: 1 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            m
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            metacritic_apiby melroy89
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            PHP Metacritic API - Mirror from my GitLab
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            PHPdotstar image 55dotUpdated: 2 y agodot License: Permissive (Apache-2.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      L
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Last-Statement-of-Death-Rowby wansho
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Last-Statement-of-Death-Row, 人之将死，其言也善
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Pythondotstar image 54dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                j
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                just-a-spiderby Jiezhi
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                🕸Just let spider save your time.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Pythondotstar image 54dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          D
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Daily-codeby rui7157
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          日常代码爬虫、gui小工具等
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Pythondotstar image 54dotUpdated: 4 y agodot License: Strong Copyleft (GPL-3.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    s
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    simple_bank_koreaby Beomi
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    simple crawler for Korean banks with Transactions
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Pythondotstar image 54dotUpdated: 2 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              T
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              TrustlookWannaCryToolkitby apkjet
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              WannaCryToolkit scanner and removal toolkit
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Pythondotstar image 54dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        s
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        spiderby binglansky
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        一些小爬虫 : )
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Pythondotstar image 54dotUpdated: 3 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  c
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  corpreditby LongYosef
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  国家企业信用信息官网爬虫，未获取全部企业信息，重点在设计反爬思路
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Pythondotstar image 54dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            c
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            crawlersby teampopong
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Some quick 'n dirty web crawlers.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Pythondotstar image 54dotUpdated: 4 y agodot License: Strong Copyleft (AGPL-3.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      b
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      browsercrawlerby spullara
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Crawl websites from your browser and save them in S3
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      JavaScriptdotstar image 54dotUpdated: 5 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                w
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                workerman-filemonitorby walkor
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                监控文件更新并自动reload workerman
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                PHPdotstar image 54dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          c
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          color-extractorby marekm4
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Simple image color extractor written in Go with no external dependencies
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Godotstar image 54dotUpdated: 2 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    n
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    nmaptableby jgamblin
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Transform NMap Scans to an D3.js HTML Table
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    HTMLdotstar image 54dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              t
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              tool-ginby bajins
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              基于go-gin框架建立减少冗余动作项目，如：下载一些工具
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Godotstar image 54dotUpdated: 2 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        c
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        crawlBaiduWenkuby vict-cn
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        这可能是爬百度文库最全的项目了
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Pythondotstar image 53dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  W
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  WebCrawlerby Misterhex
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Just a simple web crawler which return crawled links as IObservable using reactive extension and async await.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  C#dotstar image 53dotUpdated: 3 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            c
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            crawlzoneby crawlzone
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Crawlzone is a fast asynchronous internet crawling framework for PHP.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            PHPdotstar image 53dotUpdated: 3 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      b
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      box-spiderby Laichj
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      SpringBoot快速开发的爬虫项目，轻易爬取A股5000+支股票250日行情百万数据，也支持腾讯新闻、凤凰资讯数据爬取
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Javadotstar image 53dotUpdated: 2 y agodot License: Permissive (Apache-2.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                b
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                btletby neoql
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Some toolkits implements part of BT Protocol, like DHT spider.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Godotstar image 53dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          g
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          go-mysql-syncerby hacktmz
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          基于go-mysql & gorail二次开发的binlog解析工具，使用MQ跨机房同步，增加了修改表结构后依然能正常工作的特性，多个mysql同时处理的功能
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Godotstar image 53dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    I
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    ImageCrawlby dxsooo
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Web Image Crawler by scrapy
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Pythondotstar image 52dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              d
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              doubanMovieCrawlerby luckterry7
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              doubanMovieCrawler,for collecting lastest movie
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Javadotstar image 52dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        n
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        news_spiderby jfzhang95
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        新闻爬虫 (腾讯,网易,新浪,今日头条,搜狐,凤凰网,腾讯滚动新闻)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Pythondotstar image 52dotUpdated: 2 y agodot License: Permissive (Apache-2.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  p
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  proxycrawl-pythonby proxycrawl
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  ProxyCrawl Python library for scraping and crawling
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Pythondotstar image 52dotUpdated: 2 y agodot License: Permissive (Apache-2.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            n
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            netcrawlby ytti
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            LLDP/CDP crawler
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Rubydotstar image 52dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      S
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      SkyScraperby JonCanning
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      An asynchronous web scraper / web crawler using async / await and Reactive Extensions
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      C#dotstar image 52dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                u
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                url-crawlerby pop-os
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Rust crate for configurable parallel web crawling, designed to crawl for content
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Rustdotstar image 52dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Open Weaver – Develop Applications Faster with Open Source

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          kandi

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Community and Support

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Company

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Follow

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          • © 2023 Open Weaver Inc.