spider | 自动抓取duxiu.com的新闻 - 自动抓取duxiu

I have a problem with a Scrapy Python program I'm trying to build. The code is the following.

I have the following scrapy CrawlSpider:

I am working on certain stock-related projects where I have had a task to scrape all data on a daily basis for the last 5 years. i.e from 2016 to date. I particularly thought of using selenium because I can use crawler and bot to scrape the data based on the date. So I used the use of button click with selenium and now I want the same data that is displayed by the selenium browser to be fed by scrappy. This is the website I am working on right now. I have written the following code inside scrappy spider.

I have an app that store data of multiple users in stormdb for example

User A Login and send some requests and i will log it in his table And User B Login and send some requests and i will log it in his table

the issue in here when both send requests the database log user request one for all users

code example

Hello I am a new coder but I am having troubles trying to add a new user (req) to a json file of users I have for a fake bank. I'm more confused on the type of object these are and how to access them from data. I would appreciate any words of advice. Currently I can only add the new account in, but it replaces the other users in the file. So now I'm trying to add the old users and the new users before I import but I am unsure how. My main area on confusion is how to access and manipulate the "data" from fs.readfile

I borrowed the R code from the link and produced the following graph:

Using the same idea, I tried with my data as follows:

Example of book that I want to delete from the books_library (depends on the book name (input)):

{'Books': [{"Book's ID": {'001'}, "Book's Name": {'Avengers'}, "Book's Authors": {'Stan Lee'}, "Book's Published year": {'1938'}, "Book's Type": {'1'}, "Book's Copies": {'10'}}, {"Book's ID": {'002'}, "Book's Name": {'Spider Man'}, "Book's Authors": {'Stan Lee'}, "Book's Published year": {'1948'}, "Book's Type": {'2'}, "Book's Copies": {'15'}}]}

The function i'm using:

I'm trying to create a simple Scrapy function which will loop through a set of standard URLs and pull their Alexa Rank. The output I want is just two columns: One showing the scraped Alexa Rank, and one showing the URL which was scraped.

Everything seems to be working except that I cannot get the scraped URL to display correctly in my output. My code currently is:

In my scrapy code I'm trying to yield the following figures from parliament's website where all the members of parliament (MPs) are listed. Opening the links for each MP, I'm making parallel requests to get the figures I'm trying to count. I'm intending to yield each three figures below in the company of the name and the party of the MP

Here are the figures I'm trying to scrape

1. How many bill proposals that each MP has their signature on
2. How many question proposals that each MP has their signature on
3. How many times that each MP spoke on the parliament

In order to count and yield out how many bills has each member of parliament has their signature on, I'm trying to write a scraper on the members of parliament which works with 3 layers:

• Starting with the link where all MPs are listed
• From (1) accessing the individual page of each MP where the three information defined above is displayed
• 3a) Requesting the page with bill proposals and counting the number of them by len function 3b) Requesting the page with question proposals and counting the number of them by len function 3c) Requesting the page with speeches and counting the number of them by len function

What I want: I want to yield the inquiries of 3a,3b,3c with the name and the party of the MP in the same raw

• Problem 1) When I get an output to csv it only creates fields of speech count, name, part. It doesn't show me the fields of bill proposals and question proposals

• Problem 2) There are two empty values for each MP, which I guess corresponds to the values I described above at Problem1

• Problem 3) What is the better way of restructuring my code to output the three values in the same line, rather than printing each MP three times for each value that I'm scraping