scrappy | Scrappy is a fast and high-level web scraper | Scraper library

I've created two files, each with 100,000 div elements. The first is slow.html:

I am trying to download the 24-month data from www1.nseindia.com and it fails on Chrome and Firefox drivers. It just freezes after filling all the values in the required places and does not click. The webpage does not respond...

Below is the code that I am trying to execute:

I'm trying to scrape carrefour website data through python. I've used scrappy, beautiful soup, selenium but nothing seems to work. I'm getting the error that you don't have the permission to access. Is there any way to scrape this website? The code is attached below, NEED HELP!

I am currently working on a Flask app. The app takes a url from the user and then crawls that website and returns the links found in that website. This is what my code looks like:

I am trying to use python's scrappy to extract course catalog information from a website. The thing is, each course has a link to its full page and I need to iterate through those pages one by one to extract their information, which later, are fed to an SQL database. Anyhow, I don't know how to change the url's in the spider successively. here attached below is my code so far.

I have a flask app which will run a scrappy spider. The app works fine in my developement machine however when I run it in container the close method of the spider is not executed.

Here is the code to the spider:

I am trying to deploy my Docker container with Django server on Google APP Engine Custom environment, although it gets deployed but it doesn't start working the way it should work i.e it seems django runserver is not working .

app.yaml:

I have hit a wall with my tkinter built GUI wherein I am trying to have a time consuming function run on a button click that also updates several elements in my GUI at the same time. At the moment the function hasn't been built/implemented, so I am using a placeholder function that essentially just counts up to 1,000,000 (I have also used time.sleep(10) in other attempts).

The program is essentially designed to allow the user to choose an operation at the menu, and once chosen, the window changes to the operation screen and begins running the first function of that operation. Once that has completed, the user should be able to click a next button to run the next function. An indicator on the screen lets the user know which function they are on.

When I run from the menu screen however, the GUI hangs and does not update to the operation screen until the first function is complete. When I click the next button, the indicator does not update to the correct function until said function has completed.

From reading up on this, I figure my solution is going to probably involve using .after() or threading, however I have attempted to use both these options and I cant seem to get either of them working.

Bare in mind this is minimally functional code, so its pretty scrappy, but it demonstrates the issue I am running into. The chainMeta list is an external JSON list that will contain details for external python scripts that will be designed to boot up and operate functions within docker containers.

self.test() is essentially a placeholder for the time consuming scripts that will be specific to each node. node1.txt in the chainMeta is a placeholder for one of these scripts.

I am writing a project from the Automate The Boring Stuff book. The task is the following:

Image Site Downloader

Write a program that goes to a photo-sharing site like Flickr or Imgur, searches for a category of photos, and then downloads all the resulting images. You could write a program that works with any photo site that has a search feature.

Here is my code: