tech-news | DailyTechNews
kandi X-RAY | tech-news Summary
kandi X-RAY | tech-news Summary
DailyTechNews
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of tech-news
tech-news Key Features
tech-news Examples and Code Snippets
Community Discussions
Trending Discussions on tech-news
QUESTION
I am trying to get the links, titles, authors and timestamps of the 10 articles that are on this webpage: https://www.investing.com/equities/micron-tech-news
At the time of writing,
The first article is the link is "https://www.investing.com/news/economy/take-five-swap-lines-to-the-rescue-2117042",
the title is "Take Five: Swap lines to the rescue",
the author is "By Reuters"
and the time stamp is "22 hours ago".
There are a total of 10 articles on the page. However, when I try to run beautifulsoup4 with requests, I get a bunch of articles that are not supposed to be there before the first correct article appears.
And a bunch of junk articles appear at the end as well, in the end only 4 of the links appear while the others are not supposed to be there.
I don't know why the webpage is loading like this. How can I go about doing what I want?
I'm using the following code:
...ANSWER
Answered 2020-Mar-21 at 17:48In fact it seems that there are also articles on the top of the page so you may need to look at the second parent div
with class=mediumTitle1
for instance. Also for the elements with class articleDetails
sometimes they are div
sometimes they are span
so you can use find(class_="articleDetails").findNext("span")
to extract the author to deal with both :
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install tech-news
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page