Back ArrowExplore all topics

Crawler Libraries - Page 18

S
Spider_on_Tianmao_and_Taobaoby ClericPy

Python dotstar image 62 dotVersion:Currentdot
no licences License: No License (No License)

已废弃。 Spiders on Tianmao Taobao JingDong。停止更新

Support
    Quality
      Security
        License
          Reuse
            A
            ARGUSby datawizard1337

            Python dotstar image 62 dotVersion:Currentdot
            License: Strong Copyleft (GPL-3.0)

            ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks like scraping texts or collecting hyperlinks between websites. See: https://link.springer.com/article/10.1007/s11192-020-03726-9

            Support
              Quality
                Security
                  License
                    Reuse
                      a
                      amazon-python-scrapy-scraperby ian-kerins

                      Python dotstar image 62 dotVersion:Currentdot
                      no licences License: No License (No License)

                      Python Scrapy spider that scrapes all Amazon products from a keyword search

                      Support
                        Quality
                          Security
                            License
                              Reuse
                                S
                                StarWarsIntroCreatorby KasselLabs

                                JavaScript dotstar image 62 dotVersion:Currentdot
                                no licences License: No License (No License)

                                A website to create your own Star Wars opening crawl.

                                Support
                                  Quality
                                    Security
                                      License
                                        Reuse
                                          M
                                          MovieCrawlerby liguobao

                                          C# dotstar image 62 dotVersion:Currentdot
                                          no licences License: No License (No License)

                                          Movie Crawler

                                          Support
                                            Quality
                                              Security
                                                License
                                                  Reuse
                                                    1
                                                    163Musicby yokonsan

                                                    Python dotstar image 62 dotVersion:Currentdot
                                                    no licences License: No License (No License)

                                                    163music spider by scrapy.

                                                    Support
                                                      Quality
                                                        Security
                                                          License
                                                            Reuse
                                                              n
                                                              nebula_docby threathunterX

                                                              CSS dotstar image 62 dotVersion:Currentdot
                                                              License: Permissive (Apache-2.0)

                                                              nebula 文档, gitbook版

                                                              Support
                                                                Quality
                                                                  Security
                                                                    License
                                                                      Reuse
                                                                        P
                                                                        ProxyIpPoolby javagaorui5944

                                                                        Java dotstar image 61 dotVersion:Currentdot
                                                                        no licences License: No License (No License)

                                                                        :bullettrain_side:The Crawler Proxy IP Pool Component

                                                                        Support
                                                                          Quality
                                                                            Security
                                                                              License
                                                                                Reuse
                                                                                  A
                                                                                  Auto_Shadowsocksby VonSdite

                                                                                  Python dotstar image 61 dotVersion:Currentdot
                                                                                  no licences License: No License (No License)

                                                                                  Shadowsocks. 科学上网, 仅供学习。是免费的服务器，可能存在科学上网不稳定。

                                                                                  Support
                                                                                    Quality
                                                                                      Security
                                                                                        License
                                                                                          Reuse
                                                                                            d
                                                                                            douyin_spiderby ErisYoung

                                                                                            Python dotstar image 61 dotVersion:Currentdot
                                                                                            License: Permissive (MIT)

                                                                                            🎨One simple and easy to use crawler for DouYin（一个简单易用的抖音爬虫,可下载指定用户,挑战,音乐的视频,音频和数据)

                                                                                            Support
                                                                                              Quality
                                                                                                Security
                                                                                                  License
                                                                                                    Reuse
                                                                                                      H
                                                                                                      Hands-on-WebScrapingby amitupreti

                                                                                                      Python dotstar image 61 dotVersion:Currentdot
                                                                                                      License: Permissive (MIT)

                                                                                                      This repo is a part of blog series on several web scraping projects where we will explore scraping techniques to crawl data from simple websites to websites using advanced protection.

                                                                                                      Support
                                                                                                        Quality
                                                                                                          Security
                                                                                                            License
                                                                                                              Reuse
                                                                                                                M
                                                                                                                MaoYanby Germey

                                                                                                                Python dotstar image 61 dotVersion:Currentdot
                                                                                                                no licences License: No License (No License)

                                                                                                                MaoYan Top100 Spider

                                                                                                                Support
                                                                                                                  Quality
                                                                                                                    Security
                                                                                                                      License
                                                                                                                        Reuse
                                                                                                                          s
                                                                                                                          scrapy-crawl-onceby TeamHG-Memex

                                                                                                                          Python dotstar image 61 dotVersion:Currentdot
                                                                                                                          License: Permissive (MIT)

                                                                                                                          Scrapy middleware which allows to crawl only new content

                                                                                                                          Support
                                                                                                                            Quality
                                                                                                                              Security
                                                                                                                                License
                                                                                                                                  Reuse
                                                                                                                                    b
                                                                                                                                    bilibili-video-information-spiderby zhang0peter

                                                                                                                                    Python dotstar image 61 dotVersion:Currentdot
                                                                                                                                    License: Permissive (Apache-2.0)

                                                                                                                                    B站2千万视频信息爬虫

                                                                                                                                    Support
                                                                                                                                      Quality
                                                                                                                                        Security
                                                                                                                                          License
                                                                                                                                            Reuse
                                                                                                                                              m
                                                                                                                                              medium-crawlerby NISH1001

                                                                                                                                              Python dotstar image 61 dotVersion:Currentdot
                                                                                                                                              License: Strong Copyleft (GPL-3.0)

                                                                                                                                              A crawler for scraping posts from medium.com

                                                                                                                                              Support
                                                                                                                                                Quality
                                                                                                                                                  Security
                                                                                                                                                    License
                                                                                                                                                      Reuse
                                                                                                                                                        t
                                                                                                                                                        tieba-zhuaquby ankanch

                                                                                                                                                        Python dotstar image 61 dotVersion:Currentdot
                                                                                                                                                        License: Strong Copyleft (GPL-3.0)

                                                                                                                                                        百度贴吧分布式爬虫，用于贴吧数据挖掘。从贴吧维度和用户维度进行数据分析

                                                                                                                                                        Support
                                                                                                                                                          Quality
                                                                                                                                                            Security
                                                                                                                                                              License
                                                                                                                                                                Reuse
                                                                                                                                                                  r
                                                                                                                                                                  razbor-poletov.github.comby razbor-poletov

                                                                                                                                                                  JavaScript dotstar image 61 dotVersion:Currentdot
                                                                                                                                                                  no licences License: No License (No License)

                                                                                                                                                                  Podcast "Разбор полетов"

                                                                                                                                                                  Support
                                                                                                                                                                    Quality
                                                                                                                                                                      Security
                                                                                                                                                                        License
                                                                                                                                                                          Reuse
                                                                                                                                                                            d
                                                                                                                                                                            darcby JarryShaw

                                                                                                                                                                            Python dotstar image 61 dotVersion:Currentdot
                                                                                                                                                                            License: Permissive (BSD-3-Clause)

                                                                                                                                                                            Darkweb Crawler Project

                                                                                                                                                                            Support
                                                                                                                                                                              Quality
                                                                                                                                                                                Security
                                                                                                                                                                                  License
                                                                                                                                                                                    Reuse
                                                                                                                                                                                      O
                                                                                                                                                                                      OXID_Findby uknowsec

                                                                                                                                                                                      C++ dotstar image 61 dotVersion:Currentdot
                                                                                                                                                                                      no licences License: No License (No License)

                                                                                                                                                                                      OXID_Find by C++（多线程） 通过OXID解析器获取Windows远程主机上网卡地址

                                                                                                                                                                                      Support
                                                                                                                                                                                        Quality
                                                                                                                                                                                          Security
                                                                                                                                                                                            License
                                                                                                                                                                                              Reuse
                                                                                                                                                                                                c
                                                                                                                                                                                                car-pricesby go-crawler

                                                                                                                                                                                                Go dotstar image 61 dotVersion:Currentdot
                                                                                                                                                                                                License: Permissive (MIT)

                                                                                                                                                                                                Golang爬虫 爬取汽车之家 二手车产品库

                                                                                                                                                                                                Support
                                                                                                                                                                                                  Quality
                                                                                                                                                                                                    Security
                                                                                                                                                                                                      License
                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                          p
                                                                                                                                                                                                          pompby estin

                                                                                                                                                                                                          Python dotstar image 60 dotVersion:Currentdot
                                                                                                                                                                                                          License: Proprietary (Proprietary)

                                                                                                                                                                                                          Screen scraping and web crawling framework

                                                                                                                                                                                                          Support
                                                                                                                                                                                                            Quality
                                                                                                                                                                                                              Security
                                                                                                                                                                                                                License
                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                    p
                                                                                                                                                                                                                    python-piscesby wolfhong

                                                                                                                                                                                                                    Python dotstar image 60 dotVersion:Currentdot
                                                                                                                                                                                                                    License: Permissive (MIT)

                                                                                                                                                                                                                    A tools to search for and download images by keywords using search engines: google/baidu/yahoo/bing. 使用google等搜索引擎搜索关键词并下载图片

                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                          License
                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                              1
                                                                                                                                                                                                                              163Musicby Blackyukun

                                                                                                                                                                                                                              Python dotstar image 60 dotVersion:Currentdot
                                                                                                                                                                                                                              no licences License: No License (No License)

                                                                                                                                                                                                                              163music spider by scrapy.

                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                        w
                                                                                                                                                                                                                                        warta-scrapby harryandriyan

                                                                                                                                                                                                                                        Python dotstar image 60 dotVersion:Currentdot
                                                                                                                                                                                                                                        no licences License: No License (No License)

                                                                                                                                                                                                                                        Indonesia Index News Crawler, including 10 online media

                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                  l
                                                                                                                                                                                                                                                  liulishenshe_crawlerby risingsun1412

                                                                                                                                                                                                                                                  Python dotstar image 60 dotVersion:Currentdot
                                                                                                                                                                                                                                                  no licences License: No License (No License)

                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                            Z
                                                                                                                                                                                                                                                            Zhihu-Daily-Readerby nonoroazoro

                                                                                                                                                                                                                                                            JavaScript dotstar image 60 dotVersion:Currentdot
                                                                                                                                                                                                                                                            License: Proprietary (Proprietary)

                                                                                                                                                                                                                                                            :cyclone: Zhihu Daily Reader (Web).

                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                      b
                                                                                                                                                                                                                                                                      backbone-mobileby linksgo2011

                                                                                                                                                                                                                                                                      JavaScript dotstar image 60 dotVersion:Currentdot
                                                                                                                                                                                                                                                                      no licences License: No License (No License)

                                                                                                                                                                                                                                                                      mobile webapp seed project with backbone

                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                b
                                                                                                                                                                                                                                                                                bathyscapheby darkspot-org

                                                                                                                                                                                                                                                                                Go dotstar image 60 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                License: Strong Copyleft (GPL-3.0)

                                                                                                                                                                                                                                                                                Fast, highly configurable, cloud native dark web crawler.

                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                          i
                                                                                                                                                                                                                                                                                          imagecrawlerby testrain

                                                                                                                                                                                                                                                                                          Shell dotstar image 60 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                          no licences License: No License (No License)

                                                                                                                                                                                                                                                                                          An image crawler implemented in shell script

                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                    D
                                                                                                                                                                                                                                                                                                    DanmakuRenderby SmallPeaches

                                                                                                                                                                                                                                                                                                    Python dotstar image 60 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                    no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                    一个可以录制带弹幕直播流的小工具

                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                              c
                                                                                                                                                                                                                                                                                                              crawler-pythonby hezila

                                                                                                                                                                                                                                                                                                              Python dotstar image 59 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                              License: Proprietary (Proprietary)

                                                                                                                                                                                                                                                                                                              a simple crawler framework

                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                        R
                                                                                                                                                                                                                                                                                                                        RentCrawerby waylife

                                                                                                                                                                                                                                                                                                                        Python dotstar image 59 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                        License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                        Rent information crawler 租房信息抓取(北京)

                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                  b
                                                                                                                                                                                                                                                                                                                                  bcrawlby b-crawl

                                                                                                                                                                                                                                                                                                                                  C++ dotstar image 59 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                  License: Proprietary (Proprietary)

                                                                                                                                                                                                                                                                                                                                  a fork of Dungeon Crawl Stone Soup

                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                            J
                                                                                                                                                                                                                                                                                                                                            JD-SHOPPERby louisyoungx

                                                                                                                                                                                                                                                                                                                                            Python dotstar image 59 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                            License: Strong Copyleft (GPL-3.0)

                                                                                                                                                                                                                                                                                                                                            京东自动下单 (自动登录,指定时间预约商品,商品补货监控,自动加购物车,自动下单)

                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                      a
                                                                                                                                                                                                                                                                                                                                                      adult-downloadby zJiaJun

                                                                                                                                                                                                                                                                                                                                                      Java dotstar image 58 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                      no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                                      just download adult everything,enjoy

                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                i
                                                                                                                                                                                                                                                                                                                                                                itjuzi_disby hardy4yooz

                                                                                                                                                                                                                                                                                                                                                                Python dotstar image 58 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                          F
                                                                                                                                                                                                                                                                                                                                                                          Free_proxy_poolby yaleimeng

                                                                                                                                                                                                                                                                                                                                                                          Python dotstar image 58 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                          no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                                                          对免费代理IP网站进行爬取，收集汇总为自己的代理池。关键是验证代理的有效性、匿名性、去重复

                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                    s
                                                                                                                                                                                                                                                                                                                                                                                    scrapy-pyppeteerby lopuhin

                                                                                                                                                                                                                                                                                                                                                                                    Python dotstar image 58 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                    no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                                                                    Use pyppeteer from a Scrapy spider

                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                              j
                                                                                                                                                                                                                                                                                                                                                                                              jkcrawlerby CourierKyn

                                                                                                                                                                                                                                                                                                                                                                                              Python dotstar image 58 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                              License: Strong Copyleft (GPL-3.0)

                                                                                                                                                                                                                                                                                                                                                                                              使用 Scrapy 写成的 JK 爬虫，图片源自哔哩哔哩、Tumblr、Instagram，以及微博、Twitter

                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                        p
                                                                                                                                                                                                                                                                                                                                                                                                        proxyserviceby Jwnie

                                                                                                                                                                                                                                                                                                                                                                                                        Java dotstar image 58 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                        no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                                                                                        抓取网上公开代理，维护可供爬虫使用的IP池，区分墙内墙外、http/https/socks代理。

                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                  n
                                                                                                                                                                                                                                                                                                                                                                                                                  next-url-prettifierby BDav24

                                                                                                                                                                                                                                                                                                                                                                                                                  JavaScript dotstar image 58 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                  License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                                                                                                                  Url prettifier for Next Framework

                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                            L
                                                                                                                                                                                                                                                                                                                                                                                                                            LianjiaSpiderby xietongMe

                                                                                                                                                                                                                                                                                                                                                                                                                            Go dotstar image 58 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                            no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                                                                                                            Golang爬虫 爬取长沙的在售房价数据以及成交房价数据

                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                      c
                                                                                                                                                                                                                                                                                                                                                                                                                                      cuby japaric-archived

                                                                                                                                                                                                                                                                                                                                                                                                                                      Rust dotstar image 58 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                      License: Proprietary (Proprietary)

                                                                                                                                                                                                                                                                                                                                                                                                                                      Testing ground for the Copper book (http://japaric.github.io/copper/).

                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                C
                                                                                                                                                                                                                                                                                                                                                                                                                                                COUNTER-Robotsby atmire

                                                                                                                                                                                                                                                                                                                                                                                                                                                Shell dotstar image 58 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                                                                                                                                                Official list of user agents that are regarded as robots/spiders by COUNTER

                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                          A
                                                                                                                                                                                                                                                                                                                                                                                                                                                          AwsomeSpiderby kangvcar

                                                                                                                                                                                                                                                                                                                                                                                                                                                          Python dotstar image 58 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                          no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                                                                                                                                          Python爬虫小项目汇总（招聘信息/电影信息/股票信息/天气信息/贴吧信息/图片信息/视频信息..）

                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                    1
                                                                                                                                                                                                                                                                                                                                                                                                                                                                    1024Video-Crawlerby JosephPai

                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Python dotstar image 57 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                                    no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                    A python crawler for 1024 jap video from a mystery website. (No url)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                              f
                                                                                                                                                                                                                                                                                                                                                                                                                                                                              fund_crawlerby XDTD

                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Python dotstar image 57 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                                              no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                              基金爬虫，爬取天天基金的基金信息与基金经理信息

                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        c
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        crawling-projectsby guptachetan1997

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Python dotstar image 57 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Web scraping and automation using python

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  j
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  json-formatterby emmasax

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  JavaScript dotstar image 57 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Takes JSON and renders it into an HTML list.

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            p
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            posterby kingbuwu

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            JavaScript dotstar image 57 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            poster 微信小程序海报DEMO

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      S
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Spider_on_Tianmao_and_Taobaoby ClericPy
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      已废弃。 Spiders on Tianmao Taobao JingDong。停止更新
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Pythondotstar image 62dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                A
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                ARGUSby datawizard1337
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks like scraping texts or collecting hyperlinks between websites. See: https://link.springer.com/article/10.1007/s11192-020-03726-9
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Pythondotstar image 62dotUpdated: 3 y agodot License: Strong Copyleft (GPL-3.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          a
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          amazon-python-scrapy-scraperby ian-kerins
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Python Scrapy spider that scrapes all Amazon products from a keyword search
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Pythondotstar image 62dotUpdated: 2 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    S
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    StarWarsIntroCreatorby KasselLabs
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    A website to create your own Star Wars opening crawl.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    JavaScriptdotstar image 62dotUpdated: 2 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              M
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              MovieCrawlerby liguobao
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Movie Crawler
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              C#dotstar image 62dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        1
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        163Musicby yokonsan
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        163music spider by scrapy.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Pythondotstar image 62dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  n
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  nebula_docby threathunterX
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  nebula 文档, gitbook版
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  CSSdotstar image 62dotUpdated: 4 y agodot License: Permissive (Apache-2.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            P
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            ProxyIpPoolby javagaorui5944
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            :bullettrain_side:The Crawler Proxy IP Pool Component
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Javadotstar image 61dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      A
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Auto_Shadowsocksby VonSdite
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Shadowsocks. 科学上网, 仅供学习。是免费的服务器，可能存在科学上网不稳定。
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Pythondotstar image 61dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                d
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                douyin_spiderby ErisYoung
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                🎨One simple and easy to use crawler for DouYin（一个简单易用的抖音爬虫,可下载指定用户,挑战,音乐的视频,音频和数据)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Pythondotstar image 61dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          H
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Hands-on-WebScrapingby amitupreti
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          This repo is a part of blog series on several web scraping projects where we will explore scraping techniques to crawl data from simple websites to websites using advanced protection.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Pythondotstar image 61dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    M
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    MaoYanby Germey
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    MaoYan Top100 Spider
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Pythondotstar image 61dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              s
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              scrapy-crawl-onceby TeamHG-Memex
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Scrapy middleware which allows to crawl only new content
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Pythondotstar image 61dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        b
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        bilibili-video-information-spiderby zhang0peter
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        B站2千万视频信息爬虫
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Pythondotstar image 61dotUpdated: 4 y agodot License: Permissive (Apache-2.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  m
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  medium-crawlerby NISH1001
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  A crawler for scraping posts from medium.com
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Pythondotstar image 61dotUpdated: 2 y agodot License: Strong Copyleft (GPL-3.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            t
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            tieba-zhuaquby ankanch
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            百度贴吧分布式爬虫，用于贴吧数据挖掘。从贴吧维度和用户维度进行数据分析
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Pythondotstar image 61dotUpdated: 1 y agodot License: Strong Copyleft (GPL-3.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      r
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      razbor-poletov.github.comby razbor-poletov
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Podcast "Разбор полетов"
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      JavaScriptdotstar image 61dotUpdated: 2 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                d
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                darcby JarryShaw
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Darkweb Crawler Project
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Pythondotstar image 61dotUpdated: 1 y agodot License: Permissive (BSD-3-Clause)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          O
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          OXID_Findby uknowsec
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          OXID_Find by C++（多线程） 通过OXID解析器获取Windows远程主机上网卡地址
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          C++dotstar image 61dotUpdated: 3 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    c
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    car-pricesby go-crawler
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Golang爬虫 爬取汽车之家 二手车产品库
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Godotstar image 61dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              p
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              pompby estin
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Screen scraping and web crawling framework
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Pythondotstar image 60dotUpdated: 5 y agodot License: Proprietary (Proprietary)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        p
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        python-piscesby wolfhong
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        A tools to search for and download images by keywords using search engines: google/baidu/yahoo/bing. 使用google等搜索引擎搜索关键词并下载图片
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Pythondotstar image 60dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  1
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  163Musicby Blackyukun
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  163music spider by scrapy.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Pythondotstar image 60dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            w
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            warta-scrapby harryandriyan
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Indonesia Index News Crawler, including 10 online media
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Pythondotstar image 60dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      l
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      liulishenshe_crawlerby risingsun1412
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Pythondotstar image 60dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Z
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Zhihu-Daily-Readerby nonoroazoro
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                :cyclone: Zhihu Daily Reader (Web).
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                JavaScriptdotstar image 60dotUpdated: 4 y agodot License: Proprietary (Proprietary)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          b
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          backbone-mobileby linksgo2011
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          mobile webapp seed project with backbone
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          JavaScriptdotstar image 60dotUpdated: 5 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    b
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    bathyscapheby darkspot-org
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Fast, highly configurable, cloud native dark web crawler.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Godotstar image 60dotUpdated: 3 y agodot License: Strong Copyleft (GPL-3.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              i
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              imagecrawlerby testrain
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              An image crawler implemented in shell script
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Shelldotstar image 60dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        D
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        DanmakuRenderby SmallPeaches
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        一个可以录制带弹幕直播流的小工具
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Pythondotstar image 60dotUpdated: 2 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  c
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  crawler-pythonby hezila
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  a simple crawler framework
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Pythondotstar image 59dotUpdated: 4 y agodot License: Proprietary (Proprietary)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            R
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            RentCrawerby waylife
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Rent information crawler 租房信息抓取(北京)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Pythondotstar image 59dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      b
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      bcrawlby b-crawl
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      a fork of Dungeon Crawl Stone Soup
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      C++dotstar image 59dotUpdated: 2 y agodot License: Proprietary (Proprietary)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                J
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                JD-SHOPPERby louisyoungx
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                京东自动下单 (自动登录,指定时间预约商品,商品补货监控,自动加购物车,自动下单)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Pythondotstar image 59dotUpdated: 3 y agodot License: Strong Copyleft (GPL-3.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          a
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          adult-downloadby zJiaJun
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          just download adult everything,enjoy
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Javadotstar image 58dotUpdated: 3 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    i
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    itjuzi_disby hardy4yooz
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Pythondotstar image 58dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              F
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Free_proxy_poolby yaleimeng
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              对免费代理IP网站进行爬取，收集汇总为自己的代理池。关键是验证代理的有效性、匿名性、去重复
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Pythondotstar image 58dotUpdated: 3 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        s
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        scrapy-pyppeteerby lopuhin
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Use pyppeteer from a Scrapy spider
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Pythondotstar image 58dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  j
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  jkcrawlerby CourierKyn
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  使用 Scrapy 写成的 JK 爬虫，图片源自哔哩哔哩、Tumblr、Instagram，以及微博、Twitter
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Pythondotstar image 58dotUpdated: 3 y agodot License: Strong Copyleft (GPL-3.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            p
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            proxyserviceby Jwnie
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            抓取网上公开代理，维护可供爬虫使用的IP池，区分墙内墙外、http/https/socks代理。
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Javadotstar image 58dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      n
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      next-url-prettifierby BDav24
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Url prettifier for Next Framework
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      JavaScriptdotstar image 58dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                L
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                LianjiaSpiderby xietongMe
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Golang爬虫 爬取长沙的在售房价数据以及成交房价数据
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Godotstar image 58dotUpdated: 3 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          c
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          cuby japaric-archived
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Testing ground for the Copper book (http://japaric.github.io/copper/).
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Rustdotstar image 58dotUpdated: 4 y agodot License: Proprietary (Proprietary)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    C
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    COUNTER-Robotsby atmire
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Official list of user agents that are regarded as robots/spiders by COUNTER
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Shelldotstar image 58dotUpdated: 2 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              A
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              AwsomeSpiderby kangvcar
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Python爬虫小项目汇总（招聘信息/电影信息/股票信息/天气信息/贴吧信息/图片信息/视频信息..）
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Pythondotstar image 58dotUpdated: 2 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        1
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        1024Video-Crawlerby JosephPai
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        A python crawler for 1024 jap video from a mystery website. (No url)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Pythondotstar image 57dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  f
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  fund_crawlerby XDTD
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  基金爬虫，爬取天天基金的基金信息与基金经理信息
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Pythondotstar image 57dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            c
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            crawling-projectsby guptachetan1997
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Web scraping and automation using python
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Pythondotstar image 57dotUpdated: 2 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      j
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      json-formatterby emmasax
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Takes JSON and renders it into an HTML list.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      JavaScriptdotstar image 57dotUpdated: 4 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                p
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                posterby kingbuwu
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                poster 微信小程序海报DEMO
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                JavaScriptdotstar image 57dotUpdated: 4 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Open Weaver – Develop Applications Faster with Open Source

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          kandi

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Community and Support

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Company

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Follow

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          • © 2023 Open Weaver Inc.