kandi background
Explore Kits

Best Python web scraping automation libraries

by Hans Updated: May 13, 2022

Automate repetitive web data processing tasks with these Python libraries.

Working with HTTP to request a web page

requestsby psf

Python star image 47177 Version:v2.27.1

License: Permissive (Apache-2.0)

A simple, yet elegant, HTTP library.

Support
Quality
Security
License
Reuse

requestsby psf

Python star image 47177 Version:v2.27.1 License: Permissive (Apache-2.0)

A simple, yet elegant, HTTP library.
Support
Quality
Security
License
Reuse

grequestsby spyoungtech

Python star image 3774 Version:v0.6.0

License: Permissive (BSD-2-Clause)

Requests + Gevent = <3

Support
Quality
Security
License
Reuse

grequestsby spyoungtech

Python star image 3774 Version:v0.6.0 License: Permissive (BSD-2-Clause)

Requests + Gevent = <3
Support
Quality
Security
License
Reuse

httplib2by httplib2

Python star image 423 Version:Current

License: Others (Non-SPDX)

Small, fast HTTP client library for Python. Features persistent connections, cache, and Google App Engine support. Originally written by Joe Gregorio, now supported by community.

Support
Quality
Security
License
Reuse

httplib2by httplib2

Python star image 423 Version:Current License: Others (Non-SPDX)

Small, fast HTTP client library for Python. Features persistent connections, cache, and Google App Engine support. Originally written by Joe Gregorio, now supported by community.
Support
Quality
Security
License
Reuse

Complete web scraping framework

scrapyby scrapy

Python star image 42899 Version:1.8.2

License: Others (Non-SPDX)

Scrapy, a fast high-level web crawling & scraping framework for Python.

Support
Quality
Security
License
Reuse

scrapyby scrapy

Python star image 42899 Version:1.8.2 License: Others (Non-SPDX)

Scrapy, a fast high-level web crawling & scraping framework for Python.
Support
Quality
Security
License
Reuse

Parsing HTML, XML

BeautifulSoup4by il-vladislav

Python star image 75 Version:Current

License: No License (null)

BeautifulSoup 4 for Python 3.3

Support
Quality
Security
License
Reuse

BeautifulSoup4by il-vladislav

Python star image 75 Version:Current License: No License

BeautifulSoup 4 for Python 3.3
Support
Quality
Security
License
Reuse

lxmlby lxml

Python star image 2029 Version:lxml-4.8.0-1

License: Permissive (BSD-3-Clause)

The lxml XML toolkit for Python

Support
Quality
Security
License
Reuse

lxmlby lxml

Python star image 2029 Version:lxml-4.8.0-1 License: Permissive (BSD-3-Clause)

The lxml XML toolkit for Python
Support
Quality
Security
License
Reuse

See similar Kits and Libraries

Python
Automation
Networking
  • © 2022 Open Weaver Inc.