using web-scraping, database, html and javascript to build a web product.
Support
Quality
Security
License
Reuse
Maltego transform with Keskivonfer
Support
Quality
Security
License
Reuse
Scrape tweets by abusing Twitter's AJAX
Support
Quality
Security
License
Reuse
Accesses SolarGIS website and downloads pvspot data for sites
Support
Quality
Security
License
Reuse
Repository of all my web scraping tasks. This includes scraping eCommerce websites, websites with pagination, etc.
Support
Quality
Security
License
Reuse
Because web scraping shouldn't be this hard.
Support
Quality
Security
License
Reuse
Scraping made easy
Support
Quality
Security
License
Reuse
A Web Scrapper that can get info off MyAnimeList
Support
Quality
Security
License
Reuse
electronics shopping scraper
Support
Quality
Security
License
Reuse
Web scraping scripts to extract financial data
Support
Quality
Security
License
Reuse
Scrapes follower/followings info from any public instagram user and displays their connection to each other.
Support
Quality
Security
License
Reuse
this is a scraper built with python scrapy to scrap redit posts
Support
Quality
Security
License
Reuse
This repository is to publicly share scraping tools
Support
Quality
Security
License
Reuse
TEDのwebページをクローリングし、動画や字幕テキストをスクレイピング．おまけでコーパス作り(全データは取れていない)
Support
Quality
Security
License
Reuse
Scrapes all user information from vtopbeta
Support
Quality
Security
License
Reuse
s
sniff-paste_Pastebin-OSINT-Harvesterby likescam
Python 
2 Version:Current
License: No License (No License)
License: No License (No License)Support
Quality
Security
License
Reuse
python script to scrape images from source code
Support
Quality
Security
License
Reuse
Web scraping with java for fun and learning.........
Support
Quality
Security
License
Reuse
Using python to web-scrape real-time stock data
Support
Quality
Security
License
Reuse
Scrapes whois data from registrant name or email
Support
Quality
Security
License
Reuse
Prints the titles, URLs, and locations in CSV format for the second page of job listings on this site using standard libraries: [6] https://www.besmith.com/candidates/search-listings
Support
Quality
Security
License
Reuse
Support
Quality
Security
License
Reuse
A python spider to scrape jobs list and details form https://newyork.craigslist.org.
Support
Quality
Security
License
Reuse
An automated script/bot which will give magnet link and direct link to you.
Support
Quality
Security
License
Reuse
Simple Web Scrapper Developed with Python for Scrapping all the Links in the Page of given URL.
Support
Quality
Security
License
Reuse
Support
Quality
Security
License
Reuse
Instagram Post Scrapper
Support
Quality
Security
License
Reuse
Webscrpaers using Requests/Selenium package to scrape the Chinese Supreme Court Website' | ===> for summer reasearch 2020 Chinese Social Credit System;
Support
Quality
Security
License
Reuse
Support
Quality
Security
License
Reuse
Javascript Query(Jquery) that can pull out the given data from any Skillshare profile teaching page.
Support
Quality
Security
License
Reuse
My first scraper : This scraper pulls all books from books.toscrape.com and their attributes
Support
Quality
Security
License
Reuse
A tool to get lyrics for your favourite songs.
Support
Quality
Security
License
Reuse
Goal is to establish the scraping process using the Wikipedia API.
Support
Quality
Security
License
Reuse
MovieCast Scraper - Scrapes movies, tvshows and more from multiple providers
Support
Quality
Security
License
Reuse
Generate EPUBs from a .json metadata file and input HTML pages. A FOSS rewrite of setanta's "ebookmaker"
Support
Quality
Security
License
Reuse
Python BeautifulSoup/requests web scraper to crawl the Social Science Research Network (SSRN) for working papers.
Support
Quality
Security
License
Reuse
Able to scrape data in this case of a certain product on amazon. This will return all the prices, titles, and seller name's of that product within the price range that we choose.
Support
Quality
Security
License
Reuse
Dataset and MongoDB query to find my friend of a friend who is in the most Meetup groups as well as which of my friends could introduce me.
Support
Quality
Security
License
Reuse
Scrapes, with a custom throttle, GitHub username data using BeautifulSoup4 and formats the scraped data into a CSV document. DISCLAIMER: I strongly recommend to use GitHub's API instead of manually scraping the username data. Remember to respect GitHub's crawl delay guidelines, as per https://github.com/robots.txt, in order not to overload the website with requests!
Support
Quality
Security
License
Reuse
Repository of python based web scrapers for numerous websites.
Support
Quality
Security
License
Reuse
M
Multithreaded-amazon-scraperby ankushduacodes
Python 2 Version:Current License: Strong Copyleft (GPL-3.0)
Scraps the search results and generates a JSON file to store the information(like title, price, rating stars... etc) about those results.
Support
Quality
Security
License
Reuse
A tool that scrapes metadata out of mytaxi receipt pdfs.
Support
Quality
Security
License
Reuse
Scrapes county-level data to determine confirmed cases, deaths, and hospitalizations, then normalizes the data into a single model that's exported to a CSV file.
Support
Quality
Security
License
Reuse
Support
Quality
Security
License
Reuse
Support
Quality
Security
License
Reuse
Scrapes Instagram pages for new upload and tweets
Support
Quality
Security
License
Reuse
A web scraper for ArXiv, set up to find new articles, filter them by keywords, and send an email to a user with new articles that fit their interests and links to view the articles.
Support
Quality
Security
License
Reuse
Simple tutorials for web scraping in Python
Support
Quality
Security
License
Reuse
Dynamic scrapping of website using selenium
Support
Quality
Security
License
Reuse
C
Code-For-Innoplexus-Online-Hackathon-Artificial-Intelligence-AI-Challengeby PrajinkyaPimpalghare
Python 2 Version:Current License: No License (No License)
License: No License (No License)To categorize websites according to the URL and HTML data provided into 9 different categories E.g: news,profile,forumetc.
Support
Quality
Security
License
Reuse
M
Mission-to-Marsby tienl
using web-scraping, database, html and javascript to build a web product.
Python 2Updated: 4 y ago License: Permissive (MIT)
2Updated: 4 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse
k
keskivonfer-maltegoby megadose
Maltego transform with Keskivonfer
Python 2Updated: 3 y ago License: No License (No License)
2Updated: 3 y ago License: No License (No License)Support
Quality
Security
License
Reuse
t
tweet-scrapeby aopal
Scrape tweets by abusing Twitter's AJAX
Ruby 2Updated: 4 y ago License: Permissive (MIT)
2Updated: 4 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse
p
pvspot-webscrapingby teaganogorman
Accesses SolarGIS website and downloads pvspot data for sites
Python 2Updated: 5 y ago License: No License (No License)
2Updated: 5 y ago License: No License (No License)Support
Quality
Security
License
Reuse
W
Web_scrapingby Theophine
Repository of all my web scraping tasks. This includes scraping eCommerce websites, websites with pagination, etc.
Python 2Updated: 4 y ago License: No License (No License)
2Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
T
Traverseby Hedronium
Because web scraping shouldn't be this hard.
JavaScript 2Updated: 7 y ago License: No License (No License)
2Updated: 7 y ago License: No License (No License)Support
Quality
Security
License
Reuse
e
Support
Quality
Security
License
Reuse
M
MAL-Crawlerby jatinkarthik-tripathy
A Web Scrapper that can get info off MyAnimeList
Python 2Updated: 5 y ago License: No License (No License)
2Updated: 5 y ago License: No License (No License)Support
Quality
Security
License
Reuse
S
SimpleWebScraping-Javaby Hamza-Slama
electronics shopping scraper
Java 2Updated: 4 y ago License: No License (No License)
2Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
w
web-scrapingby JordiCorbilla
Web scraping scripts to extract financial data
Python 2Updated: 3 y ago License: Permissive (MIT)
2Updated: 3 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse
I
InstagramRelationshipAnalyticsby Dominik-CH
Scrapes follower/followings info from any public instagram user and displays their connection to each other.
Python 2Updated: 4 y ago License: Permissive (MIT)
2Updated: 4 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse
R
Reddit-Scraperby aish0007
this is a scraper built with python scrapy to scrap redit posts
Python 2Updated: 4 y ago License: No License (No License)
2Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
D
DataMiningby Datalators
This repository is to publicly share scraping tools
Python 2Updated: 4 y ago License: No License (No License)
2Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
T
TED_scraperby kkkodai
TEDのwebページをクローリングし、動画や字幕テキストをスクレイピング．おまけでコーパス作り(全データは取れていない)
Python 2Updated: 4 y ago License: No License (No License)
2Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
V
VIT-DATA-Scraperby Hariomagr
Scrapes all user information from vtopbeta
Python 2Updated: 6 y ago License: No License (No License)
2Updated: 6 y ago License: No License (No License)Support
Quality
Security
License
Reuse
s
sniff-paste_Pastebin-OSINT-Harvesterby likescam
Python 2Updated: 5 y ago License: No License (No License)
2Updated: 5 y ago License: No License (No License)Support
Quality
Security
License
Reuse
I
Image-Web-Scraperby rebeccabartels
python script to scrape images from source code
Python 2Updated: 4 y ago License: No License (No License)
2Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
W
Web-Scrapping-With-JSOUPby Jahidul007
Web scraping with java for fun and learning.........
Java 2Updated: 4 y ago License: Permissive (MIT)
2Updated: 4 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse
s
stockExtractorby advaitchorghade
Using python to web-scrape real-time stock data
Python 2Updated: 4 y ago License: No License (No License)
2Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
r
reverse_whoisby tonyrivera
Scrapes whois data from registrant name or email
Python 2Updated: 4 y ago License: Permissive (MIT)
2Updated: 4 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse
i
indeed_scraping_taskby logmannn
Prints the titles, URLs, and locations in CSV format for the second page of job listings on this site using standard libraries: [6] https://www.besmith.com/candidates/search-listings
Python 2Updated: 6 y ago License: No License (No License)
2Updated: 6 y ago License: No License (No License)Support
Quality
Security
License
Reuse
p
proxy-scraperby fyx0r
Python 2Updated: 3 y ago License: No License (No License)
2Updated: 3 y ago License: No License (No License)Support
Quality
Security
License
Reuse
C
Craigslist-spiderby aquatiko
A python spider to scrape jobs list and details form https://newyork.craigslist.org.
Python 2Updated: 6 y ago License: No License (No License)
2Updated: 6 y ago License: No License (No License)Support
Quality
Security
License
Reuse
M
Magnet-tronby RohtanshSehgal
An automated script/bot which will give magnet link and direct link to you.
Python 2Updated: 4 y ago License: Permissive (MIT)
2Updated: 4 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse
D
Django-Web-Scraperby arunism
Simple Web Scrapper Developed with Python for Scrapping all the Links in the Page of given URL.
Python 2Updated: 4 y ago License: No License (No License)
2Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
t
tqkc-bounty-lotteryby QuarkChain
Python 2Updated: 6 y ago License: Permissive (MIT)
2Updated: 6 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse
I
InstagramDownloaderby RynKings
Instagram Post Scrapper
Python 2Updated: 4 y ago License: No License (No License)
2Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
W
WebScraper_Research2020Sby JunjieLeiCoe
Webscrpaers using Requests/Selenium package to scrape the Chinese Supreme Court Website' | ===> for summer reasearch 2020 Chinese Social Credit System;
Python 2Updated: 3 y ago License: Permissive (MIT)
2Updated: 3 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse
S
Scrapy_YoutubeChannelby luckylion000
Python 2Updated: 4 y ago License: No License (No License)
2Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
S
Scraping-Databy shubhamrajput
Javascript Query(Jquery) that can pull out the given data from any Skillshare profile teaching page.
JavaScript 2Updated: 5 y ago License: No License (No License)
2Updated: 5 y ago License: No License (No License)Support
Quality
Security
License
Reuse
f
first_book_scraperby JackieYeates
My first scraper : This scraper pulls all books from books.toscrape.com and their attributes
Python 2Updated: 4 y ago License: No License (No License)
2Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
l
lyric-scraperby prakhar1965
A tool to get lyrics for your favourite songs.
Python 2Updated: 4 y ago License: Permissive (MIT)
2Updated: 4 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse
w
wiki_company_scrapeby NANlinear
Goal is to establish the scraping process using the Wikipedia API.
Python 2Updated: 4 y ago License: No License (No License)
2Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
s
scraperby MovieCast
MovieCast Scraper - Scrapes movies, tvshows and more from multiple providers
JavaScript 2Updated: 6 y ago License: Permissive (MIT)
2Updated: 6 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse
r
rebookmakerby shlomif
Generate EPUBs from a .json metadata file and input HTML pages. A FOSS rewrite of setanta's "ebookmaker"
Python 2Updated: 3 y ago License: Permissive (MIT)
2Updated: 3 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse
s
ssrn-scraperby karthiktadepalli1
Python BeautifulSoup/requests web scraper to crawl the Social Science Research Network (SSRN) for working papers.
Python 2Updated: 4 y ago License: No License (No License)
2Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
A
Amazon.com-Product-Scraperby hcastrio
Able to scrape data in this case of a certain product on amazon. This will return all the prices, titles, and seller name's of that product within the price range that we choose.
Python 2Updated: 4 y ago License: No License (No License)
2Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
m
meetupGraphby am-MongoDB
Dataset and MongoDB query to find my friend of a friend who is in the most Meetup groups as well as which of my friends could introduce me.
JavaScript 2Updated: 7 y ago License: No License (No License)
2Updated: 7 y ago License: No License (No License)Support
Quality
Security
License
Reuse
g
github-username-data-aggregatorby qe
Scrapes, with a custom throttle, GitHub username data using BeautifulSoup4 and formats the scraped data into a CSV document. DISCLAIMER: I strongly recommend to use GitHub's API instead of manually scraping the username data. Remember to respect GitHub's crawl delay guidelines, as per https://github.com/robots.txt, in order not to overload the website with requests!
Python 2Updated: 3 y ago License: Permissive (MIT)
2Updated: 3 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse
S
Scrapersby smash-96
Repository of python based web scrapers for numerous websites.
Python 2Updated: 4 y ago License: No License (No License)
2Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
M
Multithreaded-amazon-scraperby ankushduacodes
Scraps the search results and generates a JSON file to store the information(like title, price, rating stars... etc) about those results.
Python 2Updated: 4 y ago License: Strong Copyleft (GPL-3.0)
2Updated: 4 y ago License: Strong Copyleft (GPL-3.0)Support
Quality
Security
License
Reuse
m
mytaxi-scraperby najiji
A tool that scrapes metadata out of mytaxi receipt pdfs.
Python 2Updated: 5 y ago License: No License (No License)
2Updated: 5 y ago License: No License (No License)Support
Quality
Security
License
Reuse
c
covid-web-scraperby erik1066
Scrapes county-level data to determine confirmed cases, deaths, and hospitalizations, then normalizes the data into a single model that's exported to a CSV file.
Python 2Updated: 3 y ago License: Permissive (Apache-2.0)
2Updated: 3 y ago License: Permissive (Apache-2.0)Support
Quality
Security
License
Reuse
w
web-scrapingby choichoidee
Python 2Updated: 6 y ago License: No License (No License)
2Updated: 6 y ago License: No License (No License)Support
Quality
Security
License
Reuse
k
kodi-douban-scraper-2in1by abcdabcd987
Python 2Updated: 4 y ago License: Permissive (MIT)
2Updated: 4 y ago License: Permissive (MIT)Support
Quality
Security
License
Reuse
I
Instagram-Tweetby Sayak9495
Scrapes Instagram pages for new upload and tweets
Python 2Updated: 5 y ago License: Proprietary (Proprietary)
2Updated: 5 y ago License: Proprietary (Proprietary)Support
Quality
Security
License
Reuse
P
Paper_Finderby KPHippe
A web scraper for ArXiv, set up to find new articles, filter them by keywords, and send an email to a user with new articles that fit their interests and links to view the articles.
Python 2Updated: 4 y ago License: No License (No License)
2Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
s
scrapersby sidharthrajaram
Simple tutorials for web scraping in Python
Python 2Updated: 3 y ago License: No License (No License)
2Updated: 3 y ago License: No License (No License)Support
Quality
Security
License
Reuse
D
Dynamic_website_scrappingby Kunal614
Dynamic scrapping of website using selenium
Python 2Updated: 4 y ago License: No License (No License)
2Updated: 4 y ago License: No License (No License)Support
Quality
Security
License
Reuse
C
Code-For-Innoplexus-Online-Hackathon-Artificial-Intelligence-AI-Challengeby PrajinkyaPimpalghare
To categorize websites according to the URL and HTML data provided into 9 different categories E.g: news,profile,forumetc.
Python 2Updated: 5 y ago License: No License (No License)
2Updated: 5 y ago License: No License (No License)Support
Quality
Security
License
Reuse