DeepMind-Teaching-Machines-to-Read-and-Comprehend | Teaching Machines to Read and Comprehend | Machine Learning library
kandi X-RAY | DeepMind-Teaching-Machines-to-Read-and-Comprehend Summary
kandi X-RAY | DeepMind-Teaching-Machines-to-Read-and-Comprehend Summary
DeepMind : Teaching Machines to Read and Comprehend.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
- This function is called when the method is called
- Load parameters from pickle file
- Save the model parameters to file
DeepMind-Teaching-Machines-to-Read-and-Comprehend Key Features
DeepMind-Teaching-Machines-to-Read-and-Comprehend Examples and Code Snippets
Community Discussions
Trending Discussions on DeepMind-Teaching-Machines-to-Read-and-Comprehend
QUESTION
According to the API of tf.contrib.rnn.DropoutWrapper
:
output_keep_prob
: unit Tensor or float between 0 and 1, output keep probability; if it is constant and 1, no output dropout will be added.state_keep_prob
: unit Tensor or float between 0 and 1, output keep probability; if it is constant and 1, no output dropout will be added. State dropout is performed on the output states of the cell.
the description of these two parameters are almost the same, right?
I set output_keep_prob
as default and state_keep_prob=0.2
, the loss
is always around 11.3
after 400 mini-batches' training, while I set output_keep_prob=0.2
and state_keep_prob
as default, the loss
returned by my model quickly down to around 6.0
after 20 mini-batches! It cost me 4 days to find this bug, really magic, can anyone explain the difference between these two parameters? Thanks a lot!
hyper parameters:
- lr = 5E-4
- batch_size = 32
- state_size = 256
- multirnn_depth = 2
Here is the dataset.
...ANSWER
Answered 2017-Aug-14 at 15:22state_keep_prob
is the dropout added to the RNN's hidden states. The dropout added to the state of time stepi
will influence the calculation of statesi+1, i+2, ...
. As you have discovered, this propagation effect is often detrimental to the learning process.output_keep_prob
is the dropout added to the RNN's outputs, the dropout will have no effect on the calculation of the subsequent states.
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install DeepMind-Teaching-Machines-to-Read-and-Comprehend
You can use DeepMind-Teaching-Machines-to-Read-and-Comprehend like any standard Python library. You will need to make sure that you have a development environment consisting of a Python distribution including header files, a compiler, pip, and git installed. Make sure that your pip, setuptools, and wheel are up to date. When using pip it is generally recommended to install packages in a virtual environment to avoid changes to the system.
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page