neural-networks | Forecasting the Financial Times Stock Exchange 100 Index | Business library
kandi X-RAY | neural-networks Summary
kandi X-RAY | neural-networks Summary
Forecasting the Financial Times Stock Exchange 100 Index using Neural Networks
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
- Trains the network
- Adjust the weights of the Synapses
- Calculates the weight change for the hidden state
- Adjust the weights of the synapses
- Creates the Nimons
- Creates the features
- Connect the features and hidden features
- Connect the neurons to the output
- Calculate the weight change
- Calculates the weight change for the given network
- Calculates the hidden error term
- Initialises the neural network
- Creates the neurons
- Calculate the delayed output value
- This method initialises the network
- Calculate the summation of the delayed output values
- Transfers weight values to hidden nodes
- Transfers weighted values to the output neuron
- Calculates the weight change
neural-networks Key Features
neural-networks Examples and Code Snippets
Lines of Code : 136License : Non-SPDX (Apache License 2.0)
Copydef __init__(self,
cell,
input_keep_prob=1.0,
output_keep_prob=1.0,
state_keep_prob=1.0,
variational_recurrent=False,
input_size=None,
dtype=None def ctc_loss_v3(labels,
logits,
label_length,
logit_length,
logits_time_major=True,
unique=None,
blank_index=None,
name=None):
"""Comput def clip_by_global_norm(t_list, clip_norm, use_norm=None, name=None):
"""Clips values of multiple tensors by the ratio of the sum of their norms.
Given a tuple or list of tensors `t_list`, and a clipping ratio `clip_norm`,
this operation retur Community Discussions
Trending Discussions on neural-networks
QUESTION
Thank you for @Prune's critical comments on my questions.
I am trying to find the relationship between batch size and training time by using MNIST dataset.
By reading numerous questions in stackoverflow, such as this one: How does batch size impact time execution in neural networks? people said that the training time will be decreased when I use small batch size.
However, by trying out these two, I found that training with batch size == 1 takes way more time than batch size == 60,000. I set epoch as 10.
I split my MMIST dataset into 60k for the training and 10k for the testing.
This below is my code and results.
...ANSWER
Answered 2021-Mar-20 at 00:42This is a borderline question; you should still be able to extract this understanding from the basic literature ... eventually.
Your insight is exactly correct: you are measuring execution time per epoch, rather than total Time-to-Train (TTT). You have also carried the generic "smaller batches" advice ad absurdum: a batch size of 1 is almost guaranteed to be sub-optimal.
The mechanics are very simple at a macro level.
With a batch size of 60k (the entire training set), you run all 60k images through the model, average their results, and then do one back-propagation for that average result. This tends to lose the learning you can get from focusing on little-seen features.
With a batch size of 1, you run each image individually through the model, average the one result (a very simple operation :-) ), and do a back propagation. This tends to over-emphasize individual effects, especially retaining superstitious effects from each single image. It also gives too much weight to the initial assumptions of the first few images.
The most obvious effect of the tiny batch size is that you're doing 60k back-props instead of 1, so each epoch takes much longer.
Either of these approaches is an extreme case, usually absurd in application.
You need to experiment to find the "sweet spot" that gives you the fastest convergence to acceptable (near-optimal) accuracy. There are a few considerations in choosing your experimental design:
- Memory size: you want to be able to ingest the entire batch into memory at once. This allows your model to pipeline reading and processing. If you exceed available memory, you will lose a lot of time to swapping. If you under-use the memory, you leave some potential performance untapped.
- Processors: if you're on a multi-processor chip, you want to keep them all busy. If you care to assign processors through your OS controls, you'll also want to play with how many to assign to model computation, and how many to assign to I/O and system use. For instance, in one project I did, our group found that our 32 cores were best used with 28 allocated to computation, 4 reserved for I/O and other system functions.
- Scaling: some characteristics work best in powers of 2. You may find that a batch size that is 2^n or 3 * 2^n for some n, works best, simply because of block sizes and other system allocations.
The experimental design that has worked best for me over the years is to start with a power of 2 that is roughly the square root of the training set size. For you, there's an obvious starting guess of 256. Thus, you'd run experiments at perhaps 64, 128, 256, 512, and 1024. See which ones give you the fastest convergence.
Then do one step of refinement, using that factor of 3. For instance, if you find that the best performance comes at 128, also try 96 and 192.
You will likely see very little difference between your "sweet spot" and the adjacent batch sizes; this is the nature of most complex information systems.
QUESTION
- Problem Summary
There are two .gif images in my blog post, which are breaking the responsiveness of my site, they don't seem to get resized when opened on a mobile device. Although they seem to be fine when opened from pc.
PC view:
Mobile view:
As you can see, in mobile view the two .gif images are still the same size, which breaks the responsiveness of the page. Is there a way I could solve this issue?
The syntax I've used to include the .gif in my .mdx file is-
- Config.js file of my site:
ANSWER
Answered 2021-Feb-15 at 15:34The HTML on the question's page shows that the GIF images for figure 6(a) and 6(b) are not responsive.
Here is the HTML for figure 6(a):
QUESTION
I created a FANN in PHP with the help of some examples and tutorial from geekgirljoy and based it on the ocr example from the php-fann-repo
I'm trying to create a system which tells me, based on an order number, which type of order this is.
I have crated the training data, trained and tested it, but can't get the result that I expect. I'm now at the point where random changing of parameters isn't helping anymore, and I'm not sure if my assumptions are correct in the beginning.
A little of the training data: I got 60k lines of spaced splitted binary order numbers
...ANSWER
Answered 2021-Feb-04 at 08:16Long story short, your dataset is likely too complex for such a small and simple network.
When I wrote the OCR example, and I was kind of showing off a little by "compressing" all 94 chars into a single output neuron. It's not typically done this way and certainly not with complex datasets.
Usually, you would want to dedicate an output neuron for each "class" that the network needs to identify.
Put simply, its harder for the network to learn to properly increment or decrement the output value by 0.01 on a single neuron (as is the case of my OCR ANN) than to learn to associate a dedicated output neuron / pattern with a specific class.
You can find a better example of a more typical classifier implementation in the MNIST subfolder in my repo for the OCR "family" of neural networks: https://github.com/geekgirljoy/OCR_Neural_Network
My suggestion is to redesign your ANN.
Based on your code your network looks like this:
L0: IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
L1: HHHHHHHHHHHHHHHH
L2: O
Whereas it would probably operate (classify) your data better if you redesigned it like this:
First, determine the number of distinct classes types, in the example you gave I saw 0.07 listed so I will assume there are seven different classes of order types.
So, the ANN should look like this:
L0: IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
L1: A sufficient number of "hidden" neurons
L2: OOOOOOO
Where O1 represents class 1, O2 class 2 etc...
Which means that your training data would change to something like this:
60000 32 7
0 0 0 0 0 1 1 0 0 1 1 0 1 1 0 0 1 1 1 0 1 1 1 0 0 0 1 1 0 0 1 0
1 0 0 0 0 0 0
0 0 0 0 0 1 1 0 0 1 1 0 1 1 0 0 1 1 1 0 1 1 1 0 0 0 1 1 0 1 0 0
1 0 0 0 0 0 0
0 0 0 0 0 1 1 0 0 1 1 0 1 1 0 0 1 1 1 0 1 1 1 0 0 0 1 1 0 1 1 0
1 0 0 0 0 0 0
0 0 0 0 0 1 1 0 0 1 1 0 1 1 0 0 1 1 1 0 1 1 1 0 0 0 1 1 1 0 0 0
1 0 0 0 0 0 0
0 0 0 1 1 1 0 1 1 1 1 0 0 1 0 0 0 1 0 0 1 1 1 0 0 1 0 0 1 0 1 0
0 0 0 0 0 0 1
0 0 0 1 1 1 0 1 1 1 1 0 0 1 0 0 0 1 0 0 1 1 1 0 0 1 0 0 1 1 0 0
0 0 0 0 0 0 1
0 0 0 1 1 1 0 1 1 1 1 0 0 1 0 0 0 1 0 0 1 1 1 0 0 1 0 0 1 1 1 0
0 0 0 0 0 0 1
0 0 0 1 1 1 0 1 1 1 1 0 0 1 0 0 0 1 0 0 1 1 1 0 0 1 0 1 0 0 0 0
0 0 0 0 0 0 1
Class Output Examples:
Class 1: 1 0 0 0 0 0 0
Class 2: 0 1 0 0 0 0 0
Class 3: 0 0 1 0 0 0 0
Class 4: 0 0 0 1 0 0 0
Class 5: 0 0 0 0 1 0 0
Class 6: 0 0 0 0 0 1 0
Class 7: 0 0 0 0 0 0 1
Also, depending on your methodology, you MAY get better results using a harder negative value like -1 instead of 0, like this:
60000 32 7
-1 -1 -1 -1 -1 1 1 -1 -1 1 1 -1 1 1 -1 -1 1 1 1 -1 1 1 1 -1 -1 -1 1 1 -1 -1 1 -1
1 -1 -1 -1 -1 -1 -1
-1 -1 -1 -1 -1 1 1 -1 -1 1 1 -1 1 1 -1 -1 1 1 1 -1 1 1 1 -1 -1 -1 1 1 -1 1 -1 -1
1 -1 -1 -1 -1 -1 -1
-1 -1 -1 -1 -1 1 1 -1 -1 1 1 -1 1 1 -1 -1 1 1 1 -1 1 1 1 -1 -1 -1 1 1 -1 1 1 -1
1 -1 -1 -1 -1 -1 -1
-1 -1 -1 -1 -1 1 1 -1 -1 1 1 -1 1 1 -1 -1 1 1 1 -1 1 1 1 -1 -1 -1 1 1 1 -1 -1 -1
1 -1 -1 -1 -1 -1 -1
-1 -1 -1 1 1 1 -1 1 1 1 1 -1 -1 1 -1 -1 -1 1 -1 -1 1 1 1 -1 -1 1 -1 -1 1 -1 1 -1
-1 -1 -1 -1 -1 -1 1
-1 -1 -1 1 1 1 -1 1 1 1 1 -1 -1 1 -1 -1 -1 1 -1 -1 1 1 1 -1 -1 1 -1 -1 1 1 -1 -1
-1 -1 -1 -1 -1 -1 1
-1 -1 -1 1 1 1 -1 1 1 1 1 -1 -1 1 -1 -1 -1 1 -1 -1 1 1 1 -1 -1 1 -1 -1 1 1 1 -1
-1 -1 -1 -1 -1 -1 1
-1 -1 -1 1 1 1 -1 1 1 1 1 -1 -1 1 -1 -1 -1 1 -1 -1 1 1 1 -1 -1 1 -1 1 -1 -1 -1 -1
-1 -1 -1 -1 -1 -1 1
This is because you are using a "symmetric" hidden/output function like FANN_SIGMOID_SYMMETRIC which is a sigmoid and so the relationship between -1 to 0 and from 0 to 1 isn't linear so you should get better/harder distinctions between classifications and potentially faster training / fewer training epochs by more strongly contrasting the inputs/outputs like this.
Anyway, once you have trained the network and run your tests, you simply take the max() output neuron as your answer.
Example:
// ANN calc inputs and store outputs in the result array
$result = fann_run($ann, $input);
// Lets say the ANN responds like this:
// [-0.9,0.1,-0.2,0.4,0.1,0.5,0.6,0.99,-0.6,0.4]
// Let's also say there are 10 outputs representing that many classes
// 0 - 9
// [0,1,2,3,4,5,6,7,8,9]
//
// Find which output contains the highest value (the prediction/classification)
$highest = max($result); // $highest now contains the value 0.99
// So to convert the highest value to a class we find the key/position in the $result array
$class = array_search($highest, $result);
var_dump($class);
// int(7)
Why? Because the 7th key (7th / 8th (depending on how you look at it) is the high value):
array(0=>0.9,
1=>0.1,
2=>-0.2,
3=>0.4,
4=>0.1,
5=>0.5,
6=>0.6,
7=>0.99,
8=>-0.6,
0=>0.4
);
In the case of multiple class types being possible at the same time, you "softmax" instead.
Hope this helps! :-)
QUESTION
I am confused on how to replicate Keras (TensorFlow) convolutions in PyTorch.
In Keras, I can do something like this. (the input size is (256, 237, 1, 21) and the output size is (256, 237, 1, 1024).
ANSWER
Answered 2021-Jan-13 at 20:34In TensorFlow, tf.keras.layers.Conv1D takes in a tensor of shape (batch_shape + (steps, input_dim)). Which means that what is commonly known as channels appears on the last axis. For instance in 2D convolution you would have (batch, height, width, channels). This is different from PyTorch where the channel dimension is right after the batch axis: torch.nn.Conv1d takes in shapes of (batch, channel, length). So you will need to permute two axes.
For torch.nn.Conv1d:
in_channelsis the number of channels in the input tensorout_channelsis the number of filters, i.e. the number of channels the output will havestridethe step size of the convolutionpaddingthe zero-padding added to both sides
In PyTorch there is no option for padding='same', you will need to choose padding correctly. Here stride=1, so padding must equal to kernel_size//2 (i.e. padding=2) in order to maintain the length of the tensor.
In your example, since x has a shape of (256, 237, 1, 21), in TensorFlow's terminology it will be considered as an input with:
- a batch shape of
(256, 237), steps=1, so the length of your 1D input is1,21input channels.
Whereas in PyTorch, x of shape (256, 237, 1, 21) would be:
- batch shape of
(256, 237), 1input channel- a length of
21.
Have kept the input in both examples below (TensorFlow vs. PyTorch) as x.shape=(256, 237, 21) assuming 256 is the batch size, 237 is the length of the input sequence, and 21 is the number of channels (i.e. the input dimension, what I see as the dimension on each timestep).
In TensorFlow:
QUESTION
I created my own docs for Read the Docs. See my repository
Some of my docs files are jupyter notebook so I used nbshpinx for it.
In my computer I installed all the dependencies and it works great when I use make html.
However, Read the docs throws the error:
ANSWER
Answered 2021-Jan-12 at 09:49Solved it! I followed this tutorial
I added in readthedocs.yml:
QUESTION
Please forgive me asking Coursera ML course specific question. Hope someone who did the couser can answer.
In Coursera ML Week 4 Multi-class Classification and Neural Networks assignment, why the weight (theta) gradient is adding (plus) the derivative instead of subtracting?
...ANSWER
Answered 2020-Dec-05 at 06:27Since the gradients are calculated by averaging the gradients across all training examples, we first "accumulate" the gradients while looping across all the training examples. We do this by summing the gradient across all training examples. So the line you highlighted with the plus is not the gradient update step. (Notice that alpha is not there as well.) It might be somewhere else. It is most likely outside of the loop from 1 to m.
Also, I am not sure when you will learn about this (I'm sure it's somewhere in the course), but you could also vectorize the code :)
QUESTION
I am working with NFL play positional tracking data where there are multiple rows per play. Such I want to organize my data as such:
x_train = [[a1,b1,c1,...],[a2,b2,c2,...],...,[an,bn,cn,...]] y_train = [y1,y2,...,yn]
Where x_train holds tracking data from a play and y_train holds the outcome of the play.
I saw examples of using imdb data for sentiment analysis with a Keras LSTM model and wanted to try the same with my tracking data. But, I am having issues formatting my x_train.
...ANSWER
Answered 2020-Nov-25 at 22:10I have worked with the Keras LSTM layer in the past, and this seems like a very interesting application of it. I would like to help, but there are many things that go into formatting data for the LSTM layer and before getting it to work properly I would like to clarify the goal of this application.
The positional play data, is that where players are located on the field?
The play outcome data, is this the results of the play i.e. yards gained/lost, passing/running play, etc.?
What are the values you hope to get out of this? (Categorical or numerical)
EDIT/Answer:
Use the .append() method on a list to add to it.
QUESTION
In the post A Comprehensive Guide to Convolutional Neural Networks — the ELI5 way, it says
A ConvNet is able to successfully capture the Spatial and Temporal dependencies in an image through the application of relevant filters. The architecture performs a better fitting to the image dataset due to the reduction in the number of parameters involved and reusability of weights.
I don't see how it reduce parameter and reuse weight. Could anyone give an example?
...ANSWER
Answered 2020-Nov-22 at 06:54Consider the filter (or kernel) in image below having 9 pixels and the image having 49 pixels.
In a fully connected layer, we'll have 9*49 = 441 weights.
While in a CNN this same filter keeps on moving (convolving) over the entire image. All pixel values in image will be multiplied with those same 9 values of filter (hence we say weights are reused). So, we need just 9 weights per filter instead of 441 in FC layer.
The job of a filter is to identify features (such as texture, lines etc), which could be anywhere in an image. So, we want to reuse this same filter over the entire image.
QUESTION
I was reading this post https://machinelearningmastery.com/time-series-prediction-lstm-recurrent-neural-networks-python-keras/ and i want draw in my mind the structure of the LSTM network. Analyzing this part of the code:
...ANSWER
Answered 2020-Nov-17 at 14:09No, you still have one LSTM layer with four LSTM Neurons.
BTW: If you're looking for a fast way to visualize an ANN: Netron
QUESTION
I'm working in a project that isolate vocal parts from an audio. I'm using the DSD100 dataset, but for doing tests I'm using the DSD100subset dataset from I only use the mixtures and the vocals. I'm basing this work on this article
First I process the audios to extract a spectrogram and put it on a list, with all the audios forming four lists (trainMixed, trainVocals, testMixed, testVocals). Like this:
...ANSWER
Answered 2020-Nov-09 at 00:18It's probably an issue with specifying input data to Keras' fit() function. I would recommend using a tf.data.Dataset as input to fit() like so:
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install neural-networks
BiasNeuron.java
ContextNeuron.java
ContextSynapse.java
DataProcessor.java
HiddenNeuron.java
InputNeuron.java
MLP.java
Network.java
Neuron.java
OutputFile.java
OutputNeuron.java
RNN.java
Synapse.java
TDNN.java
Test.java
XMLParser.java
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page
