pointerNetwork | custom layer in keras to implement a pointer | Machine Learning library
kandi X-RAY | pointerNetwork Summary
kandi X-RAY | pointerNetwork Summary
a custom layer in keras to implement a pointer network decoder.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
- Calculate the crossentropy crossentropy
- Normalize an array
- Calculate the cross entropy for a given epoch
- Compute the crossentropy crossentropy
- Preprocess input files
- Split the data into train and test sets
- Sort the input sequence
pointerNetwork Key Features
pointerNetwork Examples and Code Snippets
Community Discussions
Trending Discussions on pointerNetwork
QUESTION
I want to implement Q&A systems with attention mechanism. I have two inputs; context
and query
which shapes are (batch_size, context_seq_len, embd_size)
and (batch_size, query_seq_len, embd_size)
.
I am following the below paper.
Machine Comprehension Using Match-LSTM and Answer Pointer. https://arxiv.org/abs/1608.07905
Then, I want to obtain a attention matrix which shape is (batch_size, context_seq_len, query_seq_len, embd_size)
. In the thesis, they calculate values for each row (it means each context word, G_i, alpha_i in the paper).
My code is below and it is running. But I am not sure my way is good or not. For example, I use for loop
for generating sequence data (for i in range(T):
). And to obtain each row, I use in-place operator like G[:,i,:,:]
, embd_context[:,i,:].clone()
is a good manner in pytorch? If not, where should I change the code?
And if you notice other points, let me know. I am a new in this field and pytorch. Sorry for my ambiguous question.
...ANSWER
Answered 2017-Nov-23 at 17:26I think your code is fine. You can't avoid the loop: for i in range(T):
because in equation (2) in the paper (https://openreview.net/pdf?id=B1-q5Pqxl), there is a hidden state coming from Match-LSTM cell which is involved in computing G_i
and alpha_i
vector and they are used to compute the input for next timestep of the Match-LSTM. So, you need to run the loop for every timestep of the Match-LSTM, I don't see an alternative to avoid the for
loop anyway.
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install pointerNetwork
You can use pointerNetwork like any standard Python library. You will need to make sure that you have a development environment consisting of a Python distribution including header files, a compiler, pip, and git installed. Make sure that your pip, setuptools, and wheel are up to date. When using pip it is generally recommended to install packages in a virtual environment to avoid changes to the system.
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page