udacity-deep-learning | Udacity Deep Learning class with TensorFlow | Machine Learning library
kandi X-RAY | udacity-deep-learning Summary
kandi X-RAY | udacity-deep-learning Summary
Assignments for Udacity Deep Learning class with TensorFlow in PURE Python, not IPython Notebook
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
- Merge trainable dataset
- Create nb_arrays
- Return the next batch
- Returns the id of a character
- Trains logistic regression
- Displays a sample
- Pick a list of images
- Load a single letter file
- Display the number of images in a pickle file
- Check if all images are well balanced
- Create a Seq2Sequence model
- Returns the next batch
- Build dataset
- Return the ID of a character
- Plot a sample dataset
- Find overlap between two datasets
- Extract data from a file
- Sanitize two datasets
- Generate a batch of data
- Plot embeddings
- Generate fake label
- Display overlap plot
- Plot the balance
- Try to download file
- Plots samples in each folder
udacity-deep-learning Key Features
udacity-deep-learning Examples and Code Snippets
Community Discussions
Trending Discussions on udacity-deep-learning
QUESTION
I am working on a deep-learning course from udacity and there is this one line of code that I do not understand as what it does? and why?. Can anyone help me understand it? It will be great if anyone can share any document related to it.
...ANSWER
Answered 2021-Mar-17 at 10:49What you have in dataset is a tensor (or a multidimensional array). In your case it has 3 dimensions.
QUESTION
Source code link: https://github.com/rndbrtrnd/udacity-deep-learning/blob/master/1_notmnist.ipynb
I am working with some of the existing code already written to read image's data as part of deep-learning course using ndimage.imread which is depricated and I changed it to imageio.imread to proceed. But I am still facing some issues (I think it is related to bad image's data).
Initial Code to read image data using ndimage.imread (depreciated). This code does not work as ndimage is depricated from 1.2.0 version of SciPy.
...ANSWER
Answered 2021-Mar-16 at 00:20I spent a whole day to figure out a way to handle it and it turns out the solution is simpler than I thought.
Solution: I was able to solve this issue by catching the ValueError exception and continue with the reading of image files.
Here is the code:
QUESTION
Part of code that I do not understand, how it work?
...ANSWER
Answered 2021-Mar-14 at 05:21I read it like this:
Join path and folder name which is coming from the for
loop one at a time, if the joined path and folder name is a directory. If not, just skip the said path.
The purpose of the code is to build a list of sorted directories.
QUESTION
I understand that bias are required in small networks, to shift the activation function. But in the case of Deep network that has multiple layers of CNN, pooling, dropout and other non -linear activations, is Bias really making a difference? The convolutional filter is learning local features and for a given conv output channel same bias is used.
This is not a dupe of this link. The above link only explains role of bias in small neural network and does not attempt to explain role of bias in deep-networks containing multiple CNN layers, drop-outs, pooling and non-linear activation functions.
I ran a simple experiment and the results indicated that removing bias from conv layer made no difference in final test accuracy. There are two models trained and the test-accuracy is almost same (slightly better in one without bias.)
- model_with_bias,
- model_without_bias( bias not added in conv layer)
Are they being used only for historical reasons?
If using bias provides no gain in accuracy, shouldn't we omit them? Less parameters to learn.
I would be thankful if someone who have deeper knowledge than me, could explain the significance(if- any) of these bias in deep networks.
Here is the complete code and the experiment result bias-VS-no_bias experiment
...ANSWER
Answered 2020-Feb-10 at 09:53Biases are tuned alongside weights by learning algorithms such as gradient descent. biases differ from weights is that they are independent of the output from previous layers. Conceptually bias is caused by input from a neuron with a fixed activation of 1, and so is updated by subtracting the just the product of the delta value and learning rate.
In a large model, removing the bias inputs makes very little difference because each node can make a bias node out of the average activation of all of its inputs, which by the law of large numbers will be roughly normal. At the first layer, the ability for this to happens depends on your input distribution. For MNIST for example, the input's average activation is roughly constant. On a small network, of course you need a bias input, but on a large network, removing it makes almost no difference.
Although in a large network it has no difference, it still depends on network architecture. For instance in LSTM:
Most applications of LSTMs simply initialize the LSTMs with small random weights which works well on many problems. But this initialization effectively sets the forget gate to 0.5. This introduces a vanishing gradient with a factor of 0.5 per timestep, which can cause problems whenever the long term dependencies are particularly severe. This problem is addressed by simply initializing the forget gates bias to a large value such as 1 or 2. By doing so, the forget gate will be initialized to a value that is close to 1, enabling gradient flow.
See also:
QUESTION
I'm following Udacity's LSTM tutorial but having a hard time understanding input data format of LSTM. https://github.com/rndbrtrnd/udacity-deep-learning/blob/master/6_lstm.ipynb
Can someone explain what num_unrolling in below code? or how to generate a training batch for the LSTM model?
...ANSWER
Answered 2017-Jul-27 at 07:24Remember the purpose of the RNN you are training is to predict the next character in a string and you have one LSTM for each character position. In the code above the cited code they have also made a mapping from a character to a number ' ' is 0, 'a' is 1, 'b' is 2 etc. And this is further translated into '1-hot' encodings that is ' ' which is 0 is encoded as [1 0 0 0 ... 0], 'a' which is 1 as [0 1 0 0 ... 0], 'b' as [0 0 1 0 0 ... 0]. In my explanation below I skip this mapping for clarity, so all my characters should really be numbers or actually 1-hot encodings.
Let me start with the simpler case, where batch_size = 1 and num_unrollings =1. Let us also say your training data is "anarchists advocate social relations based upon voluntary association of autonomous individuals mutu"
In this case your first character is the 'a' in anarchists and the expected output (label) is the 'n'. In the code this is represented by the return value of next(). batches = [ [ 'a' ], ['n' ]], where the first element of the list is the input and the last element is the label. This is then in step 0 fed into the RNN. In the next step the input is 'n' and the label is 'a' (the third letter in 'anarchi...', so the next batches = [ ['n'], ['a'] ] and the third step batches is batches = [ ['a'] , ['r']] and so on. Notice how the last element (self._last_batch) in the inner list is the first element in the inner list at the next time-step (batches = [self._last_batch]).
This is if the num_unrollings = 1. If num_unrollings = 5 then instead of only stepping one lstm unit forward every time you step num_unrolling=5 lstm units forward in every time-step. Thus the next function should provide the input for the 5 first RNN that is the 5 characters 'a','n','a','r','c' and also the corresponding labels 'n','a','r','c','h'. Notice the last four input characters are the same as the first 4 labels, so to be more memory efficient this is encoded as a list of the first 6 characters i.e.
batches = [ [ 'a'],['n'],['a'],['r'],['c'], ['h'] ],
with the understanding that the first 5 charaters are the input and the last 5 characters are the labels. and the next time next is called next returns the input and labels for the next 5 lstms
batches = [ [ 'h'], ['i'], ['s'], ['t'], ['s'], [' '] ] ], Notice the 'h' is also in this list as it before only was used as a label and now only is used as an input.
if batch_size > 1 you simoultaneously feed several sequences into the RNN update step. Notice here cursor is not a cursor but a list of cursors --- one for each sequence. consider now batch_size = 2. In the above example where the trainingdata is the 100 characters "anarchists advocate social relations based upon voluntary association of autonomous individuals mutu" the second sequence of text simply starts in the middle "luntary association of autonomous individuals mutu", so the batches in the first step contain the information [ 'a','n','a','r','c', 'h'] and ['l','u','n','t','a','r'], corresponding to the same 6 first characters as before and the first 6 characters after the middle. but it is organized as follows (tranposed) batches = [ [ 'a', 'l'], ['n', 'u'], ['a', 'n'], ['r', 't'], [c', 'a'], ['h', 'r']] the batches in the second time-step is contains the information [ 'h', 'i', 's', 't', 's', ' ' ] and ['r','y', ' ', 'a', 's', 's'] , but again tranposed batches = [[ 'h', 'r'], ['i', 'y'], ['s', ' '], ['t', 'a'], ['s', 's', [' ', 's'] ] and so on.
The above is a technical answer to what num_unrollings means for the generation of the batches. However num_unrollings is also the number of characters you go back in the backpropagation part of the updating of the weights in the RNN. This is so as in each time-step in the RNN learning algorithm you feed in num_unrolling input characters and you only calculate on the corresponding lstm's, while the (hidden) input from the previous part of the sequence is stored in variables that are not trainable. You may try to set the num_urollings to 1 and see whether it is harder to learn long range correlations. (you might need many timesteps).
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install udacity-deep-learning
You can use udacity-deep-learning like any standard Python library. You will need to make sure that you have a development environment consisting of a Python distribution including header files, a compiler, pip, and git installed. Make sure that your pip, setuptools, and wheel are up to date. When using pip it is generally recommended to install packages in a virtual environment to avoid changes to the system.
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page