metascraper | Scala library for scraping metadata

by lloydmeta Scala Version: v0.4.0 License: No License

X-Ray Key Features Code Snippets Community Discussions(4)Vulnerabilities Install Support

kandi X-RAY | metascraper Summary

metascraper is a Scala library. metascraper has no bugs, it has no vulnerabilities and it has low support. You can download it from GitHub.

A non-blocking Scala library that allows you to retrieve meta data from various URLs. Out of the box, supports HTML content (works with OpenGraph as well as normal HTML meta/image tags), but can be easily extended to support any kind of content (just implement Schema and SchemaFactory) so that you can work with any URL that returns an Array of Bytes. Tested against Scala 2.10.x, 2.11.x, and 2.12.x.

Support

Quality

Security

License

Reuse

Support

metascraper has a low active ecosystem.

It has 72 star(s) with 16 fork(s). There are 5 watchers for this library.

It had no major release in the last 12 months.

There are 0 open issues and 9 have been closed. On average issues are closed in 6 days. There are no pull requests.

It has a neutral sentiment in the developer community.

The latest version of metascraper is v0.4.0

Quality

metascraper has 0 bugs and 0 code smells.

Security

metascraper has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.

metascraper code analysis shows 0 unresolved vulnerabilities.

There are 0 security hotspots that need review.

License

metascraper does not have a standard license declared.

Check the repository for any license declaration and review the terms closely.

Without a license, all rights are reserved, and you cannot use the library in your applications.

Reuse

metascraper releases are available to install and integrate.

Installation instructions, examples and code snippets are available.

It has 683 lines of code, 75 functions and 18 files.

It has medium code complexity. Code complexity directly impacts maintainability of the code.

Top functions reviewed by kandi - BETA

kandi's functional review helps you automatically verify the functionalities of the libraries and avoid rework.
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of metascraper

Get all kandi verified functions for this library.

metascraper Key Features

No Key Features are available at this moment for metascraper.

metascraper Examples and Code Snippets

No Code Snippets are available at this moment for metascraper.

Community Discussions

Trending Discussions on metascraper

How to import multiple NodeJS packages together?

cors problem using nuxt in with SSR and proxy

Async NullRefrenceException from Html Agility Pack's LoadFromWebAsync

How to get Open Graph meta from URL input on the client side

QUESTION

How to import multiple NodeJS packages together?

Asked 2022-Feb-04 at 01:41

const metascraper = require('metascraper')([
  require('metascraper-author')(),
  require('metascraper-date')(),
  require('metascraper-description')(),
  require('metascraper-image')(),
  require('metascraper-logo')(),
  require('metascraper-clearbit')(),
  require('metascraper-publisher')(),
  require('metascraper-title')(),
  require('metascraper-url')()
])

...

ANSWER

Answered 2022-Feb-04 at 01:41

The following should work:

Source https://stackoverflow.com/questions/70980096

QUESTION

cors problem using nuxt in with SSR and proxy

Asked 2020-Oct-26 at 12:44

I want to make a simple wishlist with nuxt (project for fun) the idea is simple, the user add links to the main page for items he want ( ex: https://www.amazon.fr/Bracelet-ethnique-multicolore-pierres-aventurine/dp/B088FNK7WS?pd_rd_w=VcswS&pf_rd_p=93b57529-44d6-414c-8e38-22f57431b7bd&pf_rd_r=F71JXZXPJQFHJR7E6ZCB&pd_rd_r=5c6a3226-b724-44f0-9c7c-1f7d75b21c00&pd_rd_wg=AJYnZ ), links are stored to an array, ect... for a better visual, i want to prefill the name of the item and display a picture of this item based on the open graph data of the link (if available), for this usage, i found this plugin: https://metascraper.js.org#/?id=usage (work fine) to scrap the metadata of the page i target.

My problem is the following: when i do a simple this.$axios.$get(link) i have a cors issue (No** 'Access-Control-Allow-Origin' header is present on the requested resource**.) After some research, i found this error is due to Server Side Rendering and need to be solved with the usage of a proxy. But the use of @nuxt/proxy need to define one proxy per website (ex: '/amazon/': { target: 'https://amazon.com/', pathRewrite: { '^/amazon/': '' } } )

Or, i need to scrap data from any link my user put in her wishlist. Can i still use @nuxt/proxy or i need a more complex solution ?

...

ANSWER

Answered 2020-Oct-26 at 09:05

This error is actually a browser security feature, and it's triggered when you try to access a resource from a different origin, read more here

To work around it, simply fire the scrapping request from the server-side, this way you can make your server act as a proxy. And since you are using nuxt you can easily achieve this through server-middleware, thus you can create a custom API e.g: /api/meta-scraper?target=http://www.amazon.com... and from there you can fetch the needed metadata. See the below example code.

nuxt.config.js

Source https://stackoverflow.com/questions/64529365

QUESTION

Async NullRefrenceException from Html Agility Pack's LoadFromWebAsync

Asked 2020-Oct-05 at 08:31

I'm using Html Agility Pack to get meta tags from URLs. I was using HtmlWeb's Load, but I found it to be taking too long and thought that I could probably speed up performance using an async method instead. However, when I tried to use LoadFromWebAsync I get a NullReferenceException.

Using Load

...

ANSWER

Answered 2020-Oct-05 at 08:31

The issue was that I was attempting to show the data before it was loaded. This was because I was using async but not providing something to show while it was awaiting a result. It can be fixed by doing:

Source https://stackoverflow.com/questions/64202783

QUESTION

How to get Open Graph meta from URL input on the client side

Asked 2020-Jul-07 at 08:17

When an user submit a post with an URL. I want to get the Open Graph meta of the URL without a back-end server.

I've found libraries that can extract Open Graph meta such as opengraph-io and metascraper, but they all require a Node.js server to do the job.

Is it possible to achieve this on client side only?

...

ANSWER

Answered 2020-Jul-07 at 08:17

I have found a solution. Posting this answer to help anyone who is searching for a similar solution.

https://www.opengraph.io/ provides up to 100 requests a month free. You need to signup to get the API key to place into the app_id. In just few lines of code, I get the meta data in a response object:

Source https://stackoverflow.com/questions/62634812

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install metascraper

Add the following to your build.sbt. If the above does not work because it cannot be resolved, its likely because it hasn't been synced to Maven central yet. In that case, download a SNAPSHOT release of the same version by adding this to build.sbt.

Support

For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .

Find more information at: