syntaxnet | Syntaxnet Parsey McParseface wrapper for POS | Natural Language Processing library
kandi X-RAY | syntaxnet Summary
kandi X-RAY | syntaxnet Summary
When Google declared that The World’s Most Accurate Parser i.e., SyntaxNet goes open-source, it grabbed widespread attention from machine-learning developers and researchers who were interested in core applications of NLU like automatic extraction of information, translation etc. Following gif shows how syntaxnet internally builds the dependency tree:.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
- Wrapper for _apply_op .
- Inception V2d .
- Update a single step .
- Batch normalization .
- Wrapper for inception v3 .
- Imports a graph definition protobuf .
- Setup the model .
- Compile the model .
- Concatenate input tensors with space space .
- Train an optimizer .
syntaxnet Key Features
syntaxnet Examples and Code Snippets
Community Discussions
Trending Discussions on syntaxnet
QUESTION
I am trying to build a Django app that would use Keras models to make recommendations. Right now I'm trying to use one custom container that would hold both Django and Keras. Here's the Dockerfile I've written.
...ANSWER
Answered 2019-Jan-02 at 22:56It looks like tensorflow only publishes wheels (and only up to 3.6), and Alpine linux is not manylinux1
-compatible due to its use of musl
instead of glibc
. Because of this, pip
cannot find a suitable installation candidate and fails. Your best options are probably to build from source or change your base image.
QUESTION
I am new to python and
I am trying to test syntaxnet in python using this github repo.
and in the section "how to run the parser" is as following :
- git clone https://github.com/spoddutur/syntaxnet.git
- cd
- python main.py
- That's it!! It prints syntaxnet dependency parser output for given input english sentence
through some research, I understood that first one indicates that I need to install the syntaxnet package in my cmd so I did and the package was successfully installed. but I don't understand how to perform the second one what does cd do, and where and how should I use it?
also in main.py,
...ANSWER
Answered 2018-Aug-27 at 08:28The CD command means Change Directory.
Once you have finished cloning Syntaxnet Github Repository, you should enter its directory. That's why you have to enter CD command.
But have in mind that CD takes one parameter - the directory you want to enter.
In order to solve your problem you must write cd syntaxnet
resulting in:
QUESTION
I am trying to get a dependency tree in JSON format from SyntaxNet but all I get from the examples is a Sentence Object which is providing no accessors to access the parsed object or even iterate through the items listed.
When I run the examples from the docker file provided by TensorFlow/SyntaxNet, what I get is as below
...ANSWER
Answered 2018-Aug-09 at 10:01TL;DR Code at the end...
The Sentence object is an instance of the sentence_pb2.Setnence class, which is generated from protobuf definition files, specifically sentence.proto. This means that if you look at sentence.proto, you will see the fields that are defined for that class and their types. So you have a field called "tag" which is a string, a field called "label" which is a string, a field called head which is an integer and so on. In theory if you just convert to json using python's built-in functions it should work, but since protobuf classes are runtime generated metaclasses, they may produce some undesired results.
So what I did was first created a map object with all the info I wanted, then converted that to json:
QUESTION
Currently I'm trying to build syntax-aware NMT model.
In this project, I need the sequence of one of three transition actions (SHIFT, REDUCE-L, REDUCE-R)
Similar to what is in the image a
This chunk represents the transition-based dependency for 2 sentences(1 for 1 chunk split by empty lines)
I'm using Syntaxnet
to get the dependency parse tree first, but it doesn't directly provide that transition action sequences.
It's results are as follows,
Is it possible to get the action sequences similar to this image? Is it possible to convert what is achieved from this image to the original image's format.
...ANSWER
Answered 2018-Jun-06 at 14:00A function that converts a dependency tree to a sequence of transitions is called an oracle. It is a necessary component of a statistical parser. The transitions you described (shift, reduce-l, reduce-r)¹ are those of the arc-standard transition system (not the arc-eager system, which is: shift, left-arc, right-arc, reduce).
Pseudo-code for an arc-standard oracle:
QUESTION
I have a large dataset of sentences (i.e., ~5.000.000) in raw text which I want to process using SyntaxNet already trained for English. That is, I just want to process the sentences using a SyntaxNet model, I don't want to train any new model.
Setting up a processing environment with GPUs will have any effect on performance ?
I understand that most of the heavy CPU operations is on estimating the parameters and weights of the network/model, once these are estimated, applying the trained network should be faster than training.
Nevertheless, I've never worked before with Tensorflow and I don't know whether GPUs are used when one applies an already trained model to data.
Also, does anyone knows any easy way to setup SyntaxNet as a daemon or web-service, so that batch processing can be made easily?
...ANSWER
Answered 2017-Apr-22 at 22:03You still need to do a lot of tensor operations on the graph to predict something. So GPU still provides performance improvement for inference. Take a look at this nvidia paper, they have not tested their stuff on TF, but it is still relevant:
Our results show that GPUs provide state-of-the-art inference performance and energy efficiency, making them the platform of choice for anyone wanting to deploy a trained neural network in the field. In particular, the Titan X delivers between 5.3 and 6.7 times higher performance than the 16-core Xeon E5 CPU while achieving 3.6 to 4.4 times higher energy efficiency.
Regarding how to deploy your model, take a look at TF serving
QUESTION
I want to retrain existing models for SyntaxNet/DragNN and looking for some real numbers how long does it take to train models for any language (it will give me good baseline for my languages). What hardware have you used during this process?
Thank you in advance!
...ANSWER
Answered 2017-Mar-29 at 14:19it took about 24 hours on my mac pro with cpu. (10000 iterations) https://github.com/dsindex/syntaxnet
QUESTION
I exhausted my virtual memory when trying to install SyntaxNet from this Dockerfile using the Docker Toolbox. I received this message when compiling the Dockerfile:
...ANSWER
Answered 2017-Mar-02 at 12:05There are two possibilities: You could either modify the Dockerfile so that it creates a ~/.bazelrc
that contains the following text:
QUESTION
I installed bazel and upgrade it to 0.4.4 recently.
I want to try tensorflow/models/syntaxnet but it requires bazel 0.4.3.
So how can I downgrade bazel 0.4.4 to 0.4.3?
...ANSWER
Answered 2017-Feb-14 at 22:190.4.4 should work fine, too. 0.4.3 is the minimum.
If you really want, you can install 0.4.3 from the installer.
QUESTION
I know the more data, the better it's but what would be a reasonable amount of data required to train SyntaxNet?
...ANSWER
Answered 2017-Jan-29 at 18:44Based on some trial and error, I have arrived at the following minimums:
But please note that with this, I've only managed to get the steps in the NLP pipeline to run, I actually haven't managed to get anything usable out of it.
QUESTION
I annotated a corpus using pre-trained syntaxnet model (i.e. using Parse McParseface). I am having a problem understanding the output. There are two metrics reproted in the output. Are those for POS tagging and dependency parsing? If yes, which one is POS tagging performance and which one is for dependency parsing performance?
Here is the output:
INFO:tensorflow:Total processed documents: 21710
INFO:tensorflow:num correct tokens: 454150
INFO:tensorflow:total tokens: 560993
INFO:tensorflow:Seconds elapsed in evaluation: 1184.63, eval metric: 80.95%
INFO:tensorflow:Processed 206 documents
INFO:tensorflow:Total processed documents: 21710
INFO:tensorflow:num correct tokens: 291851
INFO:tensorflow:total tokens: 504496
INFO:tensorflow:Seconds elapsed in evaluation: 1193.17, eval metric: 57.85%
ANSWER
Answered 2017-Jan-26 at 15:27If you're using https://github.com/tensorflow/models/blob/master/syntaxnet/syntaxnet/demo.sh then the first metric is POS tag accuracy, the second UAS. They are only meaningful if the conll data you input contains gold POS tags and gold dependencies.
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install syntaxnet
You can use syntaxnet like any standard Python library. You will need to make sure that you have a development environment consisting of a Python distribution including header files, a compiler, pip, and git installed. Make sure that your pip, setuptools, and wheel are up to date. When using pip it is generally recommended to install packages in a virtual environment to avoid changes to the system.
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page