deep-clean | When Gradle or the IDE let | Plugin library
kandi X-RAY | deep-clean Summary
kandi X-RAY | deep-clean Summary
When Gradle or the IDE let you down, just --nuke all them caches
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of deep-clean
deep-clean Key Features
deep-clean Examples and Code Snippets
Community Discussions
Trending Discussions on deep-clean
QUESTION
I just got started with the urllib
module. I'm trying to scrape products from supermarkets and there's a website that seems to always respond with an HTTP Error 429: Too many requests
. I already did a bit of research on the Stack Overflow and no one seems to have the same problem. My code is as simple as it can get:
ANSWER
Answered 2019-Feb-17 at 02:05429 is server asking you to stop. Basically, the web server thinks you are trying to spam or scrape and it doesn't like it. Generally you should honor the server and if there is try after some time with 429 response you should follow it.
If you feel you are wrongly been asked by the server, either you can make sure that your user request is **similar" to the user request generated by an user from the browser, which will include user-agent and all the other information a regular browser would send with the request. If the server is sending you 429 despite that most probably either it has blocked your ip temporarily or permanently. In that you should look how to scrape through multiple ips.
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install deep-clean
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page