stockscraper | Scrapes Google 's stock API | REST library
kandi X-RAY | stockscraper Summary
kandi X-RAY | stockscraper Summary
A wrapper for scraping Google's stock data.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of stockscraper
stockscraper Key Features
stockscraper Examples and Code Snippets
Community Discussions
Trending Discussions on stockscraper
QUESTION
I have a program I am writing that takes user input to connect to a site, download it's html into a text, and retrieve data from a table twice a day. I understand the code will not be one size fits all for any page (I will likely "hardwire" the url into the code once I get it working). My issue presently is that my jsoup parser isn't properly reading in the tabular data. I'm not sure if my element selectors are too generic? The table looks like it is in standard table/tr/td format, but my rows array populates with size 0. If someone could help me debug my parser and possibly provide some suggestions on where to look for making it grab data silently twice a day, I'd really appreciate it! No runtime/compile errors, just need to correct output.
Source site: https://www.cnbc.com/us-markets/ Source code for table (snipet) :
...ANSWER
Answered 2020-Apr-06 at 23:17The problem here is that this site is a dynamic page that is loading content after the browser initially downloads the page. Jsoup is not going to be adequate to scrape pages like this. A couple options you have:
1) Use a tool that simulates a browser and makes all the necessary api calls. A couple options are Selenium WebDriver or HTMLUnit.
2) Figure out the api calls you are interested in on this site, and just call those api's directly to get a JSON document you can parse. You can see api details by opening developer tools in your browser, then look at the Network tab. For this site an example would be the following, which includes the stock quote for DJI:
QUESTION
I'm currently using Selenium for a simple scrape-job but are running into issues. The code throws StaleElementReferenceException at random points of execution while returning good data in others (but never completely finish)
I have been experimenting with Seleniums various waits, to no avail.
How can I improve my key loop on the urlList?
Code:
...ANSWER
Answered 2018-Aug-26 at 20:03Most important improvement will be learn how to find simple and stable locators.
Example:/html/body/div[1]/div[3]/div/div[1]/div[1]/div[3]/div/div[2]/div[1]/div[1]/div[2]/div[4]/div[2]
can be .box.with-actions:nth-child(1) .chart-data-window-item span
css selector.
Here improved but truncated code, please add required parts to it:
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install stockscraper
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page