StockScraper | Simple scraper for stock market site Bankier | Scraper library

by stovorov Python Version: Current License: No License

X-Ray Key Features Code Snippets Community Discussions(1)Vulnerabilities Install Support

kandi X-RAY | StockScraper Summary

StockScraper is a Python library typically used in Automation, Scraper, Nodejs applications. StockScraper has no bugs, it has no vulnerabilities, it has build file available and it has low support. You can download it from GitHub.

Simple scraper for stock market site Bankier.pl (maybe some others in a future)

Support

Quality

Security

License

Reuse

Support

StockScraper has a low active ecosystem.

It has 20 star(s) with 7 fork(s). There are 4 watchers for this library.

It had no major release in the last 6 months.

StockScraper has no issues reported. There are 3 open pull requests and 0 closed requests.

It has a neutral sentiment in the developer community.

The latest version of StockScraper is current.

Quality

StockScraper has 0 bugs and 0 code smells.

Security

StockScraper has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.

StockScraper code analysis shows 0 unresolved vulnerabilities.

There are 0 security hotspots that need review.

License

StockScraper does not have a standard license declared.

Check the repository for any license declaration and review the terms closely.

Without a license, all rights are reserved, and you cannot use the library in your applications.

Reuse

StockScraper releases are not available. You will need to build from source code and install.

Build file is available. You can build the component from source.

StockScraper saves you 97 person hours of effort in developing the same functionality from scratch.

It has 247 lines of code, 29 functions and 14 files.

It has medium code complexity. Code complexity directly impacts maintainability of the code.

Top functions reviewed by kandi - BETA

kandi has reviewed StockScraper and discovered the below as its top functions. This is intended to give you an instant insight into StockScraper implemented functionality, and help decide if they suit your requirements.

Get market data .
Main entry point .
Call a method on the server .
Validates HTTP response .
Convert a datetime object to UTC .
Return an HTTP client for the given client type .
Load data from config file .
Return the number of days in the given date .
Create the endpoint URL .
Convert response to JSON .

Get all kandi verified functions for this library.

StockScraper Key Features

No Key Features are available at this moment for StockScraper.

StockScraper Examples and Code Snippets

No Code Snippets are available at this moment for StockScraper.

Community Discussions

Trending Discussions on StockScraper

How to parse tabular data from CNBC Markets Page?

QUESTION

How to parse tabular data from CNBC Markets Page?

Asked 2020-Apr-06 at 23:17

I have a program I am writing that takes user input to connect to a site, download it's html into a text, and retrieve data from a table twice a day. I understand the code will not be one size fits all for any page (I will likely "hardwire" the url into the code once I get it working). My issue presently is that my jsoup parser isn't properly reading in the tabular data. I'm not sure if my element selectors are too generic? The table looks like it is in standard table/tr/td format, but my rows array populates with size 0. If someone could help me debug my parser and possibly provide some suggestions on where to look for making it grab data silently twice a day, I'd really appreciate it! No runtime/compile errors, just need to correct output.

Source site: https://www.cnbc.com/us-markets/ Source code for table (snipet) :

...

ANSWER

Answered 2020-Apr-06 at 23:17

The problem here is that this site is a dynamic page that is loading content after the browser initially downloads the page. Jsoup is not going to be adequate to scrape pages like this. A couple options you have:

1) Use a tool that simulates a browser and makes all the necessary api calls. A couple options are Selenium WebDriver or HTMLUnit.

2) Figure out the api calls you are interested in on this site, and just call those api's directly to get a JSON document you can parse. You can see api details by opening developer tools in your browser, then look at the Network tab. For this site an example would be the following, which includes the stock quote for DJI:

Source https://stackoverflow.com/questions/61070037

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install StockScraper

You can download it from GitHub.
You can use StockScraper like any standard Python library. You will need to make sure that you have a development environment consisting of a Python distribution including header files, a compiler, pip, and git installed. Make sure that your pip, setuptools, and wheel are up to date. When using pip it is generally recommended to install packages in a virtual environment to avoid changes to the system.

Support

For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .

Find more information at: