用 node.js 爬你自己的 leetcode 解题源码
Support
Quality
Security
License
Reuse
s
scrapedin-linkedin-crawlerby linkedtales
JavaScript 
177 Version:Current License: Permissive (Apache-2.0)
Crawler for LinkedIn full profiles 2019
Support
Quality
Security
License
Reuse
蝦皮非同步爬蟲 + 競品賣家分析
Support
Quality
Security
License
Reuse
Scheduler of spiders for scraping and parsing HTML and JSON pages
Support
Quality
Security
License
Reuse
疫情数据爬虫，2019新型冠状病毒数据仓库，轨迹数据，同乘数据，报道
Support
Quality
Security
License
Reuse
TouTiao Spider Demo
Support
Quality
Security
License
Reuse
抖音无水印视频下载
Support
Quality
Security
License
Reuse
Useful test spiders for Scrapy
Support
Quality
Security
License
Reuse
Tiny python web crawler
Support
Quality
Security
License
Reuse
C
Crawler-for-Github-Trendingby poozhu
JavaScript 172 Version:Current
License: No License (No License)
License: No License (No License)🕷️ A node crawler for github trending.
Support
Quality
Security
License
Reuse
使用Python3爬取煎蛋妹纸图片
Support
Quality
Security
License
Reuse
拼多多爬虫，爬取所有商品、评论等信息
Support
Quality
Security
License
Reuse
爬虫-百度百科-知识图谱探索
Support
Quality
Security
License
Reuse
Crawl some picture for fun
Support
Quality
Security
License
Reuse
Web spider as a service, spider on serverless, the engine behind kmppp.com
Support
Quality
Security
License
Reuse
Tool used to continuously monitor a Github org for mistaken public commits
Support
Quality
Security
License
Reuse
scrapy框架爬取51job(scrapy.Spider)，智联招聘(扒接口)，拉勾网(CrawlSpider)
Support
Quality
Security
License
Reuse
新浪微博爬虫(Sina weibo spider)，百度搜索结果 爬虫
Support
Quality
Security
License
Reuse
A Python implementation of Larry's famous PageRank algorithm.
Support
Quality
Security
License
Reuse
根据关键词抓取微博数据，再生成词云
Support
Quality
Security
License
Reuse
Crawling Japanese laws
Support
Quality
Security
License
Reuse
event-driven crawler implemented by C++
Support
Quality
Security
License
Reuse
A dynamic configurable news crawler based Scrapy
Support
Quality
Security
License
Reuse
Voight-Kampff is a Ruby gem that detects bots, spiders, crawlers and replicants
Support
Quality
Security
License
Reuse
一款分布式爬虫平台，帮助你更好的管理和开发爬虫。 内置一套爬虫定义规则（模版），可使用模版快速定义爬虫，也可当作框架手动开发爬虫。(兴趣使然的项目，用的不爽了就更新)
Support
Quality
Security
License
Reuse
Imgfy, create your html pages easly. :heavy_check_mark:
Support
Quality
Security
License
Reuse
抖音视频下载器，批量下载自己喜欢过的视频/上传的视频/关注用户发布的视频/关注用户喜欢的视频。当前已经无法爬取，项目暂时废弃，只能用于学习了。
Support
Quality
Security
License
Reuse
XSSCon: Simple XSS Scanner tool
Support
Quality
Security
License
Reuse
Touhou Project random music video generator/player, crawling image and video from websites to generate MV.
Support
Quality
Security
License
Reuse
基于小红书 Web 端进行的请求封装。https://reajason.github.io/xhs/
Support
Quality
Security
License
Reuse
douyu斗鱼 自动化工具 主播上线通知 & 直播视频自动录制 & 弹幕抓取
Support
Quality
Security
License
Reuse
Scrapy Training companion code
Support
Quality
Security
License
Reuse
:zap: Ayakashi.io - The next generation web scraping framework
Support
Quality
Security
License
Reuse
爬取贝壳找房，链家，安居客，58同城的房源信息，便于广大未买房子的朋友们尽快成为房奴！！！Crawl the house informations of ke.com, lianjia.com, anjvke.com, 58.com (ganji.com after the update), convenient for the majority of friends who did not buy the house as soon as to become the mortgage slave!!!
Support
Quality
Security
License
Reuse
m3u8（HLS流）下载，实现了AES解密、合并、多线程、批量下载
Support
Quality
Security
License
Reuse
外卖爬虫，定时自动抓取三大外卖平台上商家订单，平台目前包括：美团，饿了么，百度外卖
Support
Quality
Security
License
Reuse
Norconex Web Crawler (or spider) is a flexible web crawler for collecting, parsing, and manipulating data from the Internet (or Intranet) to various data repositories such as search engines.
Support
Quality
Security
License
Reuse
Loose python framework for kickass redis patterns
Support
Quality
Security
License
Reuse
avmoo.com爬虫
Support
Quality
Security
License
Reuse
d
dungeon-crawler-rpg-odby redpangilinan
JavaScript 157 Version:Current License: Strong Copyleft (GPL-3.0)
Quick dungeon crawler experience on demand with diablo inspired looting system!
Support
Quality
Security
License
Reuse
a simple distributed spider in Java. Java编写的一个简单分布式爬虫
Support
Quality
Security
License
Reuse
一个java版本的分布式的通用爬虫，可以插拔各个组件（提供默认的）
Support
Quality
Security
License
Reuse
A domain searcher named GoogleSSLdomainFinder - 基于谷歌SSL透明证书的子域名查询工具
Support
Quality
Security
License
Reuse
一个简单的python爬虫，原生python+BeautifulSoup
Support
Quality
Security
License
Reuse
c
crawler-china-mainland-universitiesby codeudan
JavaScript 156 Version:Current License: Permissive (MIT)
中国大陆大学列表爬虫
Support
Quality
Security
License
Reuse
使用playwright强力驱动的原创力文档book118和豆丁网docin下载工具
Support
Quality
Security
License
Reuse
Crawl and extract (regular or onion) webpages through TOR network
Support
Quality
Security
License
Reuse
一个浏览器端数据爬虫，做每个人的数据助手
Support
Quality
Security
License
Reuse
知乎分布式爬虫（Scrapy、Redis）
Support
Quality
Security
License
Reuse
Free proxy server, continuously crawling and providing proxies, based on Tornado and Scrapy. 免费代理服务器，基于Tornado和Scrapy，在本地搭建属于自己的代理池
Support
Quality
Security
License
Reuse
l
leetcode-spiderby Ma63d
用 node.js 爬你自己的 leetcode 解题源码
JavaScript 177Updated: 4 y ago License: No License (No License)
177Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
s
scrapedin-linkedin-crawlerby linkedtales
Crawler for LinkedIn full profiles 2019
JavaScript 177Updated: 3 y ago License: Permissive (Apache-2.0)
177Updated: 3 y ago License: Permissive (Apache-2.0)Support
Quality
Security
License
Reuse
c
crawler_shopee_publicby hsuanchi
蝦皮非同步爬蟲 + 競品賣家分析
Python 177Updated: 2 y ago License: No License (No License)
177Updated: 2 y ago License: No License (No License)Support
Quality
Security
License
Reuse
s
spiderby celrenheit
Scheduler of spiders for scraping and parsing HTML and JSON pages
Go 176Updated: 4 y ago License: Permissive (MIT)
176Updated: 4 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse
n
nCov2019_data_crawlerby LiuTianyong
疫情数据爬虫，2019新型冠状病毒数据仓库，轨迹数据，同乘数据，报道
Python 175Updated: 3 y ago License: No License (No License)
175Updated: 3 y ago License: No License (No License)Support
Quality
Security
License
Reuse
T
Support
Quality
Security
License
Reuse
p
Support
Quality
Security
License
Reuse
t
testspidersby scrapinghub
Useful test spiders for Scrapy
Python 172Updated: 4 y ago License: No License (No License)
172Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
p
Support
Quality
Security
License
Reuse
C
Crawler-for-Github-Trendingby poozhu
🕷️ A node crawler for github trending.
JavaScript 172Updated: 4 y ago License: No License (No License)
172Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
j
jandan_spiderby kulovecc
使用Python3爬取煎蛋妹纸图片
Python 171Updated: 4 y ago License: No License (No License)
171Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
p
Support
Quality
Security
License
Reuse
B
Baike-KnowledgeGraphby s-top
爬虫-百度百科-知识图谱探索
Python 171Updated: 3 y ago License: No License (No License)
171Updated: 3 y ago License: No License (No License)Support
Quality
Security
License
Reuse
f
fun_crawlerby ZhangBohan
Crawl some picture for fun
Python 169Updated: 4 y ago License: No License (No License)
169Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
s
spider-lessby slashbit
Web spider as a service, spider on serverless, the engine behind kmppp.com
JavaScript 168Updated: 4 y ago License: Permissive (MIT)
168Updated: 4 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse
g
github_commit_crawlerby jfalken
Tool used to continuously monitor a Github org for mistaken public commits
Python 167Updated: 4 y ago License: No License (No License)
167Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
J
JobSpidersby wqh0109663
scrapy框架爬取51job(scrapy.Spider)，智联招聘(扒接口)，拉勾网(CrawlSpider)
Python 167Updated: 2 y ago License: Permissive (Apache-2.0)
167Updated: 2 y ago License: Permissive (Apache-2.0)Support
Quality
Security
License
Reuse
S
Spiderby starFalll
新浪微博爬虫(Sina weibo spider)，百度搜索结果 爬虫
Python 167Updated: 4 y ago License: Permissive (MIT)
167Updated: 4 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse
P
PageRankby ashkonf
A Python implementation of Larry's famous PageRank algorithm.
Python 164Updated: 3 y ago License: Permissive (Apache-2.0)
164Updated: 3 y ago License: Permissive (Apache-2.0)Support
Quality
Security
License
Reuse
w
Support
Quality
Security
License
Reuse
l
law.e-gov.go.jpby riywo
Crawling Japanese laws
Ruby 164Updated: 4 y ago License: No License (No License)
164Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
z
zhihuCrawlerby zyearn
event-driven crawler implemented by C++
C++ 164Updated: 1 y ago License: No License (No License)
164Updated: 1 y ago License: No License (No License)Support
Quality
Security
License
Reuse
s
scrapy-dynamic-configurableby wuchong
A dynamic configurable news crawler based Scrapy
Python 163Updated: 4 y ago License: No License (No License)
163Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
V
Voight-Kampffby biola
Voight-Kampff is a Ruby gem that detects bots, spiders, crawlers and replicants
Ruby 163Updated: 4 y ago License: Permissive (MIT)
163Updated: 4 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse
y
yispiderby 2young2simple
一款分布式爬虫平台，帮助你更好的管理和开发爬虫。 内置一套爬虫定义规则（模版），可使用模版快速定义爬虫，也可当作框架手动开发爬虫。(兴趣使然的项目，用的不爽了就更新)
Go 163Updated: 2 y ago License: No License (No License)
163Updated: 2 y ago License: No License (No License)Support
Quality
Security
License
Reuse
i
imgfyby sahin
Imgfy, create your html pages easly. :heavy_check_mark:
JavaScript 162Updated: 4 y ago License: No License (No License)
162Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
d
douyin_downloaderby HeLiangHIT
抖音视频下载器，批量下载自己喜欢过的视频/上传的视频/关注用户发布的视频/关注用户喜欢的视频。当前已经无法爬取，项目暂时废弃，只能用于学习了。
Python 161Updated: 4 y ago License: Weak Copyleft (LGPL-3.0)
161Updated: 4 y ago License: Weak Copyleft (LGPL-3.0)Support
Quality
Security
License
Reuse
X
XSSConby menkrep1337
XSSCon: Simple XSS Scanner tool
Python 161Updated: 2 y ago License: Permissive (MIT)
161Updated: 2 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse
t
th-music-video-generatorby Jasonnor
Touhou Project random music video generator/player, crawling image and video from websites to generate MV.
JavaScript 161Updated: 3 y ago License: Permissive (Apache-2.0)
161Updated: 3 y ago License: Permissive (Apache-2.0)Support
Quality
Security
License
Reuse
x
xhsby ReaJason
基于小红书 Web 端进行的请求封装。https://reajason.github.io/xhs/
Python 160Updated: 1 y ago License: Permissive (MIT)
160Updated: 1 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse
p
pccoldby davidkingzyb
douyu斗鱼 自动化工具 主播上线通知 & 直播视频自动录制 & 弹幕抓取
Python 159Updated: 3 y ago License: Permissive (MIT)
159Updated: 3 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse
s
scrapy-trainingby scrapinghub
Scrapy Training companion code
Python 159Updated: 4 y ago License: Proprietary (Proprietary)
159Updated: 4 y ago License: Proprietary (Proprietary)Support
Quality
Security
License
Reuse
a
ayakashiby ayakashi-io
:zap: Ayakashi.io - The next generation web scraping framework
TypeScript 159Updated: 2 y ago License: Proprietary (Proprietary)
159Updated: 2 y ago License: Proprietary (Proprietary)Support
Quality
Security
License
Reuse
h
houseby tree-branch
爬取贝壳找房，链家，安居客，58同城的房源信息，便于广大未买房子的朋友们尽快成为房奴！！！Crawl the house informations of ke.com, lianjia.com, anjvke.com, 58.com (ganji.com after the update), convenient for the majority of friends who did not buy the house as soon as to become the mortgage slave!!!
Python 158Updated: 3 y ago License: No License (No License)
158Updated: 3 y ago License: No License (No License)Support
Quality
Security
License
Reuse
m
m3u8_downloaderby hestyle
m3u8（HLS流）下载，实现了AES解密、合并、多线程、批量下载
Python 158Updated: 2 y ago License: Permissive (Apache-2.0)
158Updated: 2 y ago License: Permissive (Apache-2.0)Support
Quality
Security
License
Reuse
w
waimai-crawlerby mudiyouyou
外卖爬虫，定时自动抓取三大外卖平台上商家订单，平台目前包括：美团，饿了么，百度外卖
HTML 158Updated: 1 y ago License: Permissive (MIT)
158Updated: 1 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse
c
collector-httpby Norconex
Norconex Web Crawler (or spider) is a flexible web crawler for collecting, parsing, and manipulating data from the Internet (or Intranet) to various data repositories such as search engines.
Java 157Updated: 1 y ago License: Permissive (Apache-2.0)
157Updated: 1 y ago License: Permissive (Apache-2.0)Support
Quality
Security
License
Reuse
k
kickass-redisby EverythingMe
Loose python framework for kickass redis patterns
Python 157Updated: 4 y ago License: No License (No License)
157Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
a
Support
Quality
Security
License
Reuse
d
dungeon-crawler-rpg-odby redpangilinan
Quick dungeon crawler experience on demand with diablo inspired looting system!
JavaScript 157Updated: 1 y ago License: Strong Copyleft (GPL-3.0)
157Updated: 1 y ago License: Strong Copyleft (GPL-3.0)Support
Quality
Security
License
Reuse
s
spiderby matuobasyouca
a simple distributed spider in Java. Java编写的一个简单分布式爬虫
Java 156Updated: 4 y ago License: No License (No License)
156Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
S
ScriptSpiderby xjtushilei
一个java版本的分布式的通用爬虫，可以插拔各个组件（提供默认的）
Java 156Updated: 4 y ago License: Permissive (Apache-2.0)
156Updated: 4 y ago License: Permissive (Apache-2.0)Support
Quality
Security
License
Reuse
G
GSDFby We5ter
A domain searcher named GoogleSSLdomainFinder - 基于谷歌SSL透明证书的子域名查询工具
Python 156Updated: 4 y ago License: Permissive (Apache-2.0)
156Updated: 4 y ago License: Permissive (Apache-2.0)Support
Quality
Security
License
Reuse
P
Pythonspiderby StephinChou
一个简单的python爬虫，原生python+BeautifulSoup
Python 156Updated: 4 y ago License: No License (No License)
156Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
c
crawler-china-mainland-universitiesby codeudan
中国大陆大学列表爬虫
JavaScript 156Updated: 2 y ago License: Permissive (MIT)
156Updated: 2 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse
d
docdownby kerm-me
使用playwright强力驱动的原创力文档book118和豆丁网docin下载工具
Python 156Updated: 2 y ago License: No License (No License)
156Updated: 2 y ago License: No License (No License)Support
Quality
Security
License
Reuse
T
TorCrawl.pyby MikeMeliz
Crawl and extract (regular or onion) webpages through TOR network
Python 155Updated: 2 y ago License: Strong Copyleft (GPL-3.0)
155Updated: 2 y ago License: Strong Copyleft (GPL-3.0)Support
Quality
Security
License
Reuse
p
Support
Quality
Security
License
Reuse
Z
Support
Quality
Security
License
Reuse
f
fp-serverby Karmenzind
Free proxy server, continuously crawling and providing proxies, based on Tornado and Scrapy. 免费代理服务器，基于Tornado和Scrapy，在本地搭建属于自己的代理池
Python 154Updated: 3 y ago License: Permissive (MIT)
154Updated: 3 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse