kandi background
Explore Kits

MNIST | Neural Network with single hidden layer learning MNIST | Machine Learning library

 by   evolvingstuff Java Version: Current License: MIT

 by   evolvingstuff Java Version: Current License: MIT

Download this library from

kandi X-RAY | MNIST Summary

MNIST is a Java library typically used in Artificial Intelligence, Machine Learning, Deep Learning, Tensorflow, Keras, Neural Network applications. MNIST has no vulnerabilities, it has a Permissive License and it has low support. However MNIST has 9 bugs and it build file is not available. You can download it from GitHub.
Neural Network with single hidden layer learning MNIST with less than 1.2% test error. This is the source code for the following blog post: http://evolvingstuff.blogspot.com/2012/12/learning-mnist-with-shallow-neural.html.
Support
Support
Quality
Quality
Security
Security
License
License
Reuse
Reuse

kandi-support Support

  • MNIST has a low active ecosystem.
  • It has 19 star(s) with 18 fork(s). There are 3 watchers for this library.
  • It had no major release in the last 12 months.
  • MNIST has no issues reported. There are no pull requests.
  • It has a neutral sentiment in the developer community.
  • The latest version of MNIST is current.
MNIST Support
Best in #Machine Learning
Average in #Machine Learning
MNIST Support
Best in #Machine Learning
Average in #Machine Learning

quality kandi Quality

  • MNIST has 9 bugs (4 blocker, 0 critical, 0 major, 5 minor) and 132 code smells.
MNIST Quality
Best in #Machine Learning
Average in #Machine Learning
MNIST Quality
Best in #Machine Learning
Average in #Machine Learning

securitySecurity

  • MNIST has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.
  • MNIST code analysis shows 0 unresolved vulnerabilities.
  • There are 1 security hotspots that need review.
MNIST Security
Best in #Machine Learning
Average in #Machine Learning
MNIST Security
Best in #Machine Learning
Average in #Machine Learning

license License

  • MNIST is licensed under the MIT License. This license is Permissive.
  • Permissive licenses have the least restrictions, and you can use them in most projects.
MNIST License
Best in #Machine Learning
Average in #Machine Learning
MNIST License
Best in #Machine Learning
Average in #Machine Learning

buildReuse

  • MNIST releases are not available. You will need to build from source code and install.
  • MNIST has no build file. You will be need to create the build yourself to build the component from source.
  • MNIST saves you 184 person hours of effort in developing the same functionality from scratch.
  • It has 455 lines of code, 43 functions and 11 files.
  • It has medium code complexity. Code complexity directly impacts maintainability of the code.
MNIST Reuse
Best in #Machine Learning
Average in #Machine Learning
MNIST Reuse
Best in #Machine Learning
Average in #Machine Learning
Top functions reviewed by kandi - BETA

kandi has reviewed MNIST and discovered the below as its top functions. This is intended to give you an instant insight into MNIST implemented functionality, and help decide if they suit your requirements.

  • Main entry point for the neural network .
    • Evaluate fitness for the user supervised agent .
      • Convert a file to a matrix .
        • Evaluate a sample of the bimg
          • Write a matrix to a file .
            • Returns the backpropagation for the given delta .
              • Calculates the given number of contacts
                • Activates the operator at x .
                  • Get the effective input dimension .
                    • Adds a hidden layer .

                      Get all kandi verified functions for this library.

                      Get all kandi verified functions for this library.

                      MNIST Key Features

                      Neural Network with single hidden layer learning MNIST with less than 1.2% test error.

                      Audio widget within Jupyter notebook is **not** playing. How can I get the widget to play the audio?

                      copy iconCopydownload iconDownload
                      %matplotlib inline
                      import hub
                      from IPython.display import display, Audio
                      from ipywidgets import interactive
                      
                      # Obtain the data using the hub module
                      ds = hub.load("hub://activeloop/spoken_mnist")
                      
                      # Create widget
                      sample = ds.audio[0].numpy()[:,0]
                      display(Audio(data=sample, rate = 8000, autoplay=True))
                      

                      How to calculate maximum gradient for each layer given a mini-batch

                      copy iconCopydownload iconDownload
                      import tensorflow as tf
                      import tensorflow_datasets as tfds 
                      
                      (ds_train, ds_test), ds_info = tfds.load(
                          'mnist',
                          split=['train', 'test'],
                          shuffle_files=True,
                          as_supervised=True,
                          with_info=True,
                      )
                      n = 5
                      act_func = 'relu'
                      
                      classifier = tf.keras.models.Sequential()
                      classifier.add(tf.keras.layers.Flatten(input_shape = (28, 28, 1)))
                      for i in range(n):
                        classifier.add(tf.keras.layers.Dense(32, activation=act_func))
                      classifier.add(tf.keras.layers.Dense(10, activation='softmax'))
                      opt = tf.keras.optimizers.SGD(learning_rate=0.01)
                      loss = tf.keras.losses.CategoricalCrossentropy()
                      
                      classifier.summary()
                      
                      epochs = 1
                      for epoch in range(epochs):
                          print("\nStart of epoch %d" % (epoch,))
                          for step, (x_batch_train, y_batch_train) in enumerate(ds_train.take(50).batch(10)):
                              x_batch_train = tf.cast(x_batch_train, dtype=tf.float32)
                              y_batch_train = tf.keras.utils.to_categorical(y_batch_train, 10)
                      
                              with tf.GradientTape() as tape:
                                logits = classifier(x_batch_train, training=True)
                                loss_value = loss(y_batch_train, logits)
                      
                              grads = tape.gradient(loss_value, classifier.trainable_weights)
                              opt.apply_gradients(zip(grads, classifier.trainable_weights)) 
                      
                              with tf.GradientTape(persistent=True) as tape:
                                tape.watch(x_batch_train)
                                x = classifier.layers[0](x_batch_train)
                                outputs = []
                                for layer in classifier.layers[1:]:
                                    x = layer(x)
                                    outputs.append(x)
                      
                              for idx, output in enumerate(outputs):
                                 grad = tf.math.abs(tape.gradient(output, x_batch_train))
                                 print('Max gradient for layer {} is {}'.format(idx + 1, tf.reduce_max(grad)))
                              print('End of batch {}'.format(step + 1))
                      
                      Model: "sequential_9"
                      _________________________________________________________________
                       Layer (type)                Output Shape              Param #   
                      =================================================================
                       flatten_9 (Flatten)         (None, 784)               0         
                                                                                       
                       dense_54 (Dense)            (None, 32)                25120     
                                                                                       
                       dense_55 (Dense)            (None, 32)                1056      
                                                                                       
                       dense_56 (Dense)            (None, 32)                1056      
                                                                                       
                       dense_57 (Dense)            (None, 32)                1056      
                                                                                       
                       dense_58 (Dense)            (None, 32)                1056      
                                                                                       
                       dense_59 (Dense)            (None, 10)                330       
                                                                                       
                      =================================================================
                      Total params: 29,674
                      Trainable params: 29,674
                      Non-trainable params: 0
                      _________________________________________________________________
                      
                      Start of epoch 0
                      Max gradient for layer 1 is 0.7913536429405212
                      Max gradient for layer 2 is 0.8477020859718323
                      Max gradient for layer 3 is 0.7188305854797363
                      Max gradient for layer 4 is 0.5108454823493958
                      Max gradient for layer 5 is 0.3362882435321808
                      Max gradient for layer 6 is 1.9748875867975357e-09
                      End of batch 1
                      Max gradient for layer 1 is 0.7535678148269653
                      Max gradient for layer 2 is 0.6814548373222351
                      Max gradient for layer 3 is 0.5748667120933533
                      Max gradient for layer 4 is 0.5439972877502441
                      Max gradient for layer 5 is 0.27793681621551514
                      Max gradient for layer 6 is 1.9541412932255753e-09
                      End of batch 2
                      Max gradient for layer 1 is 0.8606255650520325
                      Max gradient for layer 2 is 0.8506941795349121
                      Max gradient for layer 3 is 0.8556670546531677
                      Max gradient for layer 4 is 0.43756356835365295
                      Max gradient for layer 5 is 0.2675274908542633
                      Max gradient for layer 6 is 3.7072431791074223e-09
                      End of batch 3
                      Max gradient for layer 1 is 0.7640039324760437
                      Max gradient for layer 2 is 0.6926062107086182
                      Max gradient for layer 3 is 0.6164448857307434
                      Max gradient for layer 4 is 0.43013691902160645
                      Max gradient for layer 5 is 0.32356566190719604
                      Max gradient for layer 6 is 3.2926392723453546e-09
                      End of batch 4
                      Max gradient for layer 1 is 0.7604862451553345
                      Max gradient for layer 2 is 0.6908300518989563
                      Max gradient for layer 3 is 0.6122230887413025
                      Max gradient for layer 4 is 0.39982378482818604
                      Max gradient for layer 5 is 0.3172021210193634
                      Max gradient for layer 6 is 2.3238742041797877e-09
                      End of batch 5
                      
                      import tensorflow as tf
                      import tensorflow_datasets as tfds 
                      
                      (ds_train, ds_test), ds_info = tfds.load(
                          'mnist',
                          split=['train', 'test'],
                          shuffle_files=True,
                          as_supervised=True,
                          with_info=True,
                      )
                      n = 5
                      act_func = 'relu'
                      
                      classifier = tf.keras.models.Sequential()
                      classifier.add(tf.keras.layers.Flatten(input_shape = (28, 28, 1)))
                      for i in range(n):
                        classifier.add(tf.keras.layers.Dense(32, activation=act_func))
                      classifier.add(tf.keras.layers.Dense(10, activation='softmax'))
                      opt = tf.keras.optimizers.SGD(learning_rate=0.01)
                      loss = tf.keras.losses.CategoricalCrossentropy()
                      
                      classifier.summary()
                      
                      epochs = 1
                      for epoch in range(epochs):
                          print("\nStart of epoch %d" % (epoch,))
                          for step, (x_batch_train, y_batch_train) in enumerate(ds_train.take(50).batch(10)):
                              x_batch_train = tf.cast(x_batch_train, dtype=tf.float32)
                              y_batch_train = tf.keras.utils.to_categorical(y_batch_train, 10)
                      
                              with tf.GradientTape() as tape:
                                logits = classifier(x_batch_train, training=True)
                                loss_value = loss(y_batch_train, logits)
                      
                              grads = tape.gradient(loss_value, classifier.trainable_weights)
                              opt.apply_gradients(zip(grads, classifier.trainable_weights)) 
                      
                              with tf.GradientTape(persistent=True) as tape:
                                tape.watch(x_batch_train)
                                x = classifier.layers[0](x_batch_train)
                                outputs = []
                                for layer in classifier.layers[1:]:
                                    x = layer(x)
                                    outputs.append(x)
                      
                              for idx, output in enumerate(outputs):
                                 grad = tf.math.abs(tape.gradient(output, x_batch_train))
                                 print('Max gradient for layer {} is {}'.format(idx + 1, tf.reduce_max(grad)))
                              print('End of batch {}'.format(step + 1))
                      
                      Model: "sequential_9"
                      _________________________________________________________________
                       Layer (type)                Output Shape              Param #   
                      =================================================================
                       flatten_9 (Flatten)         (None, 784)               0         
                                                                                       
                       dense_54 (Dense)            (None, 32)                25120     
                                                                                       
                       dense_55 (Dense)            (None, 32)                1056      
                                                                                       
                       dense_56 (Dense)            (None, 32)                1056      
                                                                                       
                       dense_57 (Dense)            (None, 32)                1056      
                                                                                       
                       dense_58 (Dense)            (None, 32)                1056      
                                                                                       
                       dense_59 (Dense)            (None, 10)                330       
                                                                                       
                      =================================================================
                      Total params: 29,674
                      Trainable params: 29,674
                      Non-trainable params: 0
                      _________________________________________________________________
                      
                      Start of epoch 0
                      Max gradient for layer 1 is 0.7913536429405212
                      Max gradient for layer 2 is 0.8477020859718323
                      Max gradient for layer 3 is 0.7188305854797363
                      Max gradient for layer 4 is 0.5108454823493958
                      Max gradient for layer 5 is 0.3362882435321808
                      Max gradient for layer 6 is 1.9748875867975357e-09
                      End of batch 1
                      Max gradient for layer 1 is 0.7535678148269653
                      Max gradient for layer 2 is 0.6814548373222351
                      Max gradient for layer 3 is 0.5748667120933533
                      Max gradient for layer 4 is 0.5439972877502441
                      Max gradient for layer 5 is 0.27793681621551514
                      Max gradient for layer 6 is 1.9541412932255753e-09
                      End of batch 2
                      Max gradient for layer 1 is 0.8606255650520325
                      Max gradient for layer 2 is 0.8506941795349121
                      Max gradient for layer 3 is 0.8556670546531677
                      Max gradient for layer 4 is 0.43756356835365295
                      Max gradient for layer 5 is 0.2675274908542633
                      Max gradient for layer 6 is 3.7072431791074223e-09
                      End of batch 3
                      Max gradient for layer 1 is 0.7640039324760437
                      Max gradient for layer 2 is 0.6926062107086182
                      Max gradient for layer 3 is 0.6164448857307434
                      Max gradient for layer 4 is 0.43013691902160645
                      Max gradient for layer 5 is 0.32356566190719604
                      Max gradient for layer 6 is 3.2926392723453546e-09
                      End of batch 4
                      Max gradient for layer 1 is 0.7604862451553345
                      Max gradient for layer 2 is 0.6908300518989563
                      Max gradient for layer 3 is 0.6122230887413025
                      Max gradient for layer 4 is 0.39982378482818604
                      Max gradient for layer 5 is 0.3172021210193634
                      Max gradient for layer 6 is 2.3238742041797877e-09
                      End of batch 5
                      
                      optimizer=tf.keras.optimizers.Adam(0.001)
                      loss=tf.keras.losses.SparseCategoricalCrossentropy()
                      
                      # Iterate over the batches of a dataset.
                      for inputs, targets in zip(x_train, y_train):
                          # Open a GradientTape.
                          with tf.GradientTape() as tape:
                              # Forward pass.
                              predictions = model(inputs)
                              # Compute the loss value for this batch.
                              loss_value = loss(targets, predictions)
                          
                          # Get gradients of loss wrt the weights.
                          gradients = tape.gradient(loss_value, model.trainable_weights)
                          grads_and_vars = zip(gradients, model.trainable_weights)
                          # Update the weights of the model.
                          optimizer.apply_gradients(grads_and_vars)
                      
                      for layer in range(0, 4): # for 4 layers
                          print('max gradient of layer={}, kernel={}, bias={}'.format(
                              layer, gradients[layer].numpy().max(), gradients[layer*2+1].numpy().max()))
                      
                      optimizer=tf.keras.optimizers.Adam(0.001)
                      loss=tf.keras.losses.SparseCategoricalCrossentropy()
                      
                      # Iterate over the batches of a dataset.
                      for inputs, targets in zip(x_train, y_train):
                          # Open a GradientTape.
                          with tf.GradientTape() as tape:
                              # Forward pass.
                              predictions = model(inputs)
                              # Compute the loss value for this batch.
                              loss_value = loss(targets, predictions)
                          
                          # Get gradients of loss wrt the weights.
                          gradients = tape.gradient(loss_value, model.trainable_weights)
                          grads_and_vars = zip(gradients, model.trainable_weights)
                          # Update the weights of the model.
                          optimizer.apply_gradients(grads_and_vars)
                      
                      for layer in range(0, 4): # for 4 layers
                          print('max gradient of layer={}, kernel={}, bias={}'.format(
                              layer, gradients[layer].numpy().max(), gradients[layer*2+1].numpy().max()))
                      

                      Save GAN generated images one by one

                      copy iconCopydownload iconDownload
                      def generate_and_save_images(model, epoch, test_input):
                        predictions = model(test_input, training=False)
                        fig = plt.figure(figsize=(4, 4))
                      
                        for i in range(predictions.shape[0]):
                            plt.subplot(4, 4, i+1)
                            plt.imshow(predictions[i, :, :, 0] * 127.5 + 127.5, cmap='gray')
                            plt.imsave('image_at_epoch_{:04d}-{}.png'.format(epoch, i), predictions[i, :, :, 0] * 127.5 + 127.5, cmap='gray')
                            plt.axis('off')
                      
                        plt.savefig('image_at_epoch_{:04d}.png'.format(epoch))
                        plt.show()
                      

                      Flux.jl : Customizing optimizer

                      copy iconCopydownload iconDownload
                      for layer in model
                        for output_neuron in layer
                          for weight_element in parameters(output_neuron)
                            weight_element = sample(N(0, sqrt(2 / num_outputs(layer))))
                          end
                        end
                        sigmas[layer] = stddev(parameters(layer))
                      end
                      
                      for c in 1 to C
                        for layer in reverse(model)
                          for output_neuron in layer
                            for weight_element in parameters(output_neuron)
                              x, y = sample(batches)
                              dw = N(0, sigmas[layer])
                              # optimize weights
                            end
                          end
                        end
                      end
                      
                      using Statistics: std
                      
                      # this function will set the weights according to the
                      # normal distribution and the number of output neurons
                      # it also returns the standard deviation of the weights
                      function sample_weight!(layer::Dense)
                        sample = randn(eltype(layer.weight), size(layer.weight))
                        num_outputs = size(layer.weight, 1)
                        # notice the "." notation which is used to mutate the array
                        layer.weight .= sample .* num_outputs
                      
                        return std(layer.weight)
                      end
                      
                      function sample_weight!(layer::Conv)
                        sample = randn(eltype(layer.weight), size(layer.weight))
                        num_outputs = size(layer.weight, 4)
                        # notice the "." notation which is used to mutate the array
                        layer.weight .= sample .* num_outputs
                      
                        return std(layer.weight)
                      end
                      
                      sigmas = map(sample_weights!, model)
                      
                      function optimize_layer!(loss, layer::Dense, data, sigma)
                        for i in 1:size(layer.weight, 1)
                          for j in 1:size(layer.weight, 2)
                            wj = layer.weight[i, j]
                            x, y = data[rand(1:length(data))]
                            dw = randn() * sigma
                            ws = [wj + dw, wj, wj - dw]
                            losses = Float32[]
                            for (k, w) in enumerate(ws)
                              layer.weight[i, j] = w
                              losses[k] = loss(x, y)
                            end
                            layer.weight[i, j] = ws[argmin(losses)]
                          end
                        end
                      end
                      
                      function optimize_layer!(loss, layer::Conv, data, sigma)
                        for i in 1:size(layer.weight, 4)
                          # we use a view to reference the full kernel
                          # for this output channel
                          wid = view(layer.weight, :, :, :, i)
                          
                          # each index let's us treat wid like a vector
                          for j in eachindex(wid)
                            wj = wid[j]
                            x, y = data[rand(1:length(data))]
                            dw = randn() * sigma
                            ws = [wj + dw, wj, wj - dw]
                            losses = Float32[]
                            for (k, w) in enumerate(ws)
                              wid[j] = w
                              losses[k] = loss(x, y)
                            end
                            wid[j] = ws[argmin(losses)]
                          end
                        end
                      end
                      
                      for c in 1:C
                        for (layer, sigma) in reverse(zip(model, sigmas))
                          optimize_layer!(layer, data, sigma) do x, y
                            logitcrossentropy(model(x), y; agg = sum)
                          end
                        end
                      end
                      
                      for layer in model
                        for output_neuron in layer
                          for weight_element in parameters(output_neuron)
                            weight_element = sample(N(0, sqrt(2 / num_outputs(layer))))
                          end
                        end
                        sigmas[layer] = stddev(parameters(layer))
                      end
                      
                      for c in 1 to C
                        for layer in reverse(model)
                          for output_neuron in layer
                            for weight_element in parameters(output_neuron)
                              x, y = sample(batches)
                              dw = N(0, sigmas[layer])
                              # optimize weights
                            end
                          end
                        end
                      end
                      
                      using Statistics: std
                      
                      # this function will set the weights according to the
                      # normal distribution and the number of output neurons
                      # it also returns the standard deviation of the weights
                      function sample_weight!(layer::Dense)
                        sample = randn(eltype(layer.weight), size(layer.weight))
                        num_outputs = size(layer.weight, 1)
                        # notice the "." notation which is used to mutate the array
                        layer.weight .= sample .* num_outputs
                      
                        return std(layer.weight)
                      end
                      
                      function sample_weight!(layer::Conv)
                        sample = randn(eltype(layer.weight), size(layer.weight))
                        num_outputs = size(layer.weight, 4)
                        # notice the "." notation which is used to mutate the array
                        layer.weight .= sample .* num_outputs
                      
                        return std(layer.weight)
                      end
                      
                      sigmas = map(sample_weights!, model)
                      
                      function optimize_layer!(loss, layer::Dense, data, sigma)
                        for i in 1:size(layer.weight, 1)
                          for j in 1:size(layer.weight, 2)
                            wj = layer.weight[i, j]
                            x, y = data[rand(1:length(data))]
                            dw = randn() * sigma
                            ws = [wj + dw, wj, wj - dw]
                            losses = Float32[]
                            for (k, w) in enumerate(ws)
                              layer.weight[i, j] = w
                              losses[k] = loss(x, y)
                            end
                            layer.weight[i, j] = ws[argmin(losses)]
                          end
                        end
                      end
                      
                      function optimize_layer!(loss, layer::Conv, data, sigma)
                        for i in 1:size(layer.weight, 4)
                          # we use a view to reference the full kernel
                          # for this output channel
                          wid = view(layer.weight, :, :, :, i)
                          
                          # each index let's us treat wid like a vector
                          for j in eachindex(wid)
                            wj = wid[j]
                            x, y = data[rand(1:length(data))]
                            dw = randn() * sigma
                            ws = [wj + dw, wj, wj - dw]
                            losses = Float32[]
                            for (k, w) in enumerate(ws)
                              wid[j] = w
                              losses[k] = loss(x, y)
                            end
                            wid[j] = ws[argmin(losses)]
                          end
                        end
                      end
                      
                      for c in 1:C
                        for (layer, sigma) in reverse(zip(model, sigmas))
                          optimize_layer!(layer, data, sigma) do x, y
                            logitcrossentropy(model(x), y; agg = sum)
                          end
                        end
                      end
                      
                      for layer in model
                        for output_neuron in layer
                          for weight_element in parameters(output_neuron)
                            weight_element = sample(N(0, sqrt(2 / num_outputs(layer))))
                          end
                        end
                        sigmas[layer] = stddev(parameters(layer))
                      end
                      
                      for c in 1 to C
                        for layer in reverse(model)
                          for output_neuron in layer
                            for weight_element in parameters(output_neuron)
                              x, y = sample(batches)
                              dw = N(0, sigmas[layer])
                              # optimize weights
                            end
                          end
                        end
                      end
                      
                      using Statistics: std
                      
                      # this function will set the weights according to the
                      # normal distribution and the number of output neurons
                      # it also returns the standard deviation of the weights
                      function sample_weight!(layer::Dense)
                        sample = randn(eltype(layer.weight), size(layer.weight))
                        num_outputs = size(layer.weight, 1)
                        # notice the "." notation which is used to mutate the array
                        layer.weight .= sample .* num_outputs
                      
                        return std(layer.weight)
                      end
                      
                      function sample_weight!(layer::Conv)
                        sample = randn(eltype(layer.weight), size(layer.weight))
                        num_outputs = size(layer.weight, 4)
                        # notice the "." notation which is used to mutate the array
                        layer.weight .= sample .* num_outputs
                      
                        return std(layer.weight)
                      end
                      
                      sigmas = map(sample_weights!, model)
                      
                      function optimize_layer!(loss, layer::Dense, data, sigma)
                        for i in 1:size(layer.weight, 1)
                          for j in 1:size(layer.weight, 2)
                            wj = layer.weight[i, j]
                            x, y = data[rand(1:length(data))]
                            dw = randn() * sigma
                            ws = [wj + dw, wj, wj - dw]
                            losses = Float32[]
                            for (k, w) in enumerate(ws)
                              layer.weight[i, j] = w
                              losses[k] = loss(x, y)
                            end
                            layer.weight[i, j] = ws[argmin(losses)]
                          end
                        end
                      end
                      
                      function optimize_layer!(loss, layer::Conv, data, sigma)
                        for i in 1:size(layer.weight, 4)
                          # we use a view to reference the full kernel
                          # for this output channel
                          wid = view(layer.weight, :, :, :, i)
                          
                          # each index let's us treat wid like a vector
                          for j in eachindex(wid)
                            wj = wid[j]
                            x, y = data[rand(1:length(data))]
                            dw = randn() * sigma
                            ws = [wj + dw, wj, wj - dw]
                            losses = Float32[]
                            for (k, w) in enumerate(ws)
                              wid[j] = w
                              losses[k] = loss(x, y)
                            end
                            wid[j] = ws[argmin(losses)]
                          end
                        end
                      end
                      
                      for c in 1:C
                        for (layer, sigma) in reverse(zip(model, sigmas))
                          optimize_layer!(layer, data, sigma) do x, y
                            logitcrossentropy(model(x), y; agg = sum)
                          end
                        end
                      end
                      
                      function RSO(train_loader, test_loader, C,model, batch_size, device, args)
                      """
                      model = convolutional model structure
                      C = Number of rounds to update parameters (epochs)
                      batch_size = size of the mini batch that will be used to calculate loss
                      device = CPU or GPU
                      """
                      
                      # Evaluate initial weight
                      test_loss, test_acc = loss_and_accuracy(test_loader, model, device)
                      println("Initial Weight:")
                      println("   test_loss = $test_loss, test_accuracy = $test_acc")
                      
                      random_batch = []
                      for (x, l) in train_loader
                          push!(random_batch, (x,l))
                      end
                      
                      # Initialize weights
                      std_prep = []
                      σ_d = Float64[]
                      D = 0
                      for layer in model
                          D += 1
                          Wd = Flux.params(layer)
                          # Initialize the weights of the network with Gaussian distribution
                          for id in Wd
                              if typeof(id) == Array{Float32, 4}
                                  wj = convert(Array{Float32, 4}, rand(Normal(0, sqrt(2/length(id))), size(id)))
                              elseif typeof(id) == Vector{Float32}
                                  wj = convert(Vector{Float32}, rand(Normal(0, sqrt(2/length(id))), length(id)))
                              elseif typeof(id) == Matrix{Float32}
                                  wj = convert(Matrix{Float32}, rand(Normal(0, sqrt(2/length(id))), size(id)))
                              end
                              id = wj
                              append!(std_prep, vec(wj))
                          end
                          # Compute std of all elements in the weight tensor Wd
                          push!(σ_d, std(std_prep))
                      end
                      
                      # Weight update
                      for c in 1:C
                          d = D
                          # First update the weights of the layer closest to the labels 
                          # and then sequentially move closer to the input
                          while d > 0
                              Wd = Flux.params(model[d])
                              for id in Wd
                                  # Randomly sample change in weights from Gaussian distribution
                                  for j in 1:length(id)
                                      # Randomly sample mini-batch
                                      (x, y) = rand(random_batch, 1)[1]
                                      x, y = device(x), device(y)
                                      
                                      # Sample a weight from normal distribution
                                      ΔWj = rand(Normal(0, σ_d[d]), 1)[1]
                      
                                      # Weight update with three scenario
                                      ## F(x,l, W+ΔWj)
                                      id[j] = id[j]+ΔWj
                                      ŷ = model(x)
                                      ls_pos = logitcrossentropy(ŷ, y, agg=sum) / size(x)[end]
                      
                                      ## F(x,l,W)
                                      id[j] = id[j]-ΔWj
                                      ŷ = model(x)
                                      ls_org = logitcrossentropy(ŷ, y, agg=sum) / size(x)[end]
                      
                                      ## F(x,l, W-ΔWj)
                                      id[j] = id[j]-ΔWj
                                      ŷ = model(x)
                                      ls_neg = logitcrossentropy(ŷ, y, agg=sum) / size(x)[end]
                      
                                      # Check weight update that gives minimum loss
                                      min_loss = argmin([ls_org, ls_pos, ls_neg])
                      
                                      # Save weight update with minimum loss
                                      if min_loss == 1
                                          id[j] = id[j] + ΔWj
                                      elseif min_loss == 2
                                          id[j] = id[j] + 2*ΔWj
                                      elseif min_loss == 3
                                          id[j] = id[j]
                                      end
                          
                                  end
                              end
                              d -= 1
                          end
                      
                          train_loss, train_acc = loss_and_accuracy(train_loader, model, device)
                          test_loss, test_acc = loss_and_accuracy(test_loader, model, device)
                      
                          track!(args.tracker, test_acc)
                      
                          println("RSO Round=$c")
                          println("   train_loss = $train_loss, train_accuracy = $train_acc")
                          println("   test_loss = $test_loss, test_accuracy = $test_acc")
                      
                      end
                      
                      return Flux.params(model)
                      end
                      

                      Generator model doesn't produce pictures during training

                      copy iconCopydownload iconDownload
                      plt.imshow(predictions[i, :, :, 0]*255, cmap='gray')
                      
                      plt.imshow(predictions[i, :, :, 0], cmap='gray', vmin=0, vmax=1)
                      
                      plt.imshow(predictions[i, :, :, 0]*255, cmap='gray')
                      
                      plt.imshow(predictions[i, :, :, 0], cmap='gray', vmin=0, vmax=1)
                      

                      partial tucker decomposition

                      copy iconCopydownload iconDownload
                      """
                      Partial tucker decomposition via Higher Order Orthogonal Iteration (HOI)
                      Decomposes 'tensor' into a Tucker decomposition exclusively along 
                      the provided modes.
                      
                      Parameters
                      ----------
                      tensor: ndarray
                      modes: int list
                      list of the modes on which to perform the decomposition
                      rank: None, int or int list
                      size of the core tensor, 
                      if int, the same rank is used for all modes
                      """
                      
                      import mxnet as mx
                      import numpy as np
                      import tensorly as tl
                      import matplotlib.pyplot as plt
                      import tensorly.decomposition
                      
                      # Load data
                      mnist = mx.test_utils.get_mnist()
                      train_data = mnist['train_data'][:,0]
                      
                      
                      err = np.zeros([28,28]) # here I will save the errors for each rank
                      batch = train_data[::100] # process only 1% of the data to go faster
                      for i in range(1,28):
                        for j in range(1,28):
                          if err[i,j] == 0:
                            # Decompose the data
                            core, factors = tl.decomposition.partial_tucker(
                                              batch, modes=[1,2], tol=10e-5, rank=[i,j])
                            # Reconstruct data from features
                            c = tl.tenalg.multi_mode_dot(core, factors, modes=[1,2]);
                            # Calculate the RMS error and save
                            err[i,j] = np.sqrt(np.mean((c - batch)**2));
                      
                      # Plot the statistics
                      plt.figure(figsize=(9,6))
                      CS = plt.contour(np.log2(err), levels=np.arange(-6, 0));
                      plt.clabel(CS, CS.levels, inline=True, fmt='$2^{%d}$', fontsize=16)
                      plt.xlabel('rank 2')
                      plt.ylabel('rank 1')
                      plt.grid()
                      plt.title('Reconstruction RMS error');
                      
                      X = np.zeros([28, 28])
                      X[...] = np.nan;
                      p = 28 * 28;
                      for e,i,j in sorted([(err[i,j], i, j) for i in range(1, 28) for j in range(1, 28)]):
                        if p < i * j:
                          # we can achieve this error with some better compression
                          pass
                        else:
                          p = i * j;
                          X[i,j] = e;
                      plt.imshow(X)
                      
                      import mxnet as mx
                      import numpy as np
                      import tensorly as tl
                      import matplotlib.pyplot as plt
                      import tensorly.decomposition
                      
                      # Load data
                      mnist = mx.test_utils.get_mnist()
                      train_data = mnist['train_data'][:,0]
                      
                      
                      err = np.zeros([28,28]) # here I will save the errors for each rank
                      batch = train_data[::100] # process only 1% of the data to go faster
                      for i in range(1,28):
                        for j in range(1,28):
                          if err[i,j] == 0:
                            # Decompose the data
                            core, factors = tl.decomposition.partial_tucker(
                                              batch, modes=[1,2], tol=10e-5, rank=[i,j])
                            # Reconstruct data from features
                            c = tl.tenalg.multi_mode_dot(core, factors, modes=[1,2]);
                            # Calculate the RMS error and save
                            err[i,j] = np.sqrt(np.mean((c - batch)**2));
                      
                      # Plot the statistics
                      plt.figure(figsize=(9,6))
                      CS = plt.contour(np.log2(err), levels=np.arange(-6, 0));
                      plt.clabel(CS, CS.levels, inline=True, fmt='$2^{%d}$', fontsize=16)
                      plt.xlabel('rank 2')
                      plt.ylabel('rank 1')
                      plt.grid()
                      plt.title('Reconstruction RMS error');
                      
                      X = np.zeros([28, 28])
                      X[...] = np.nan;
                      p = 28 * 28;
                      for e,i,j in sorted([(err[i,j], i, j) for i in range(1, 28) for j in range(1, 28)]):
                        if p < i * j:
                          # we can achieve this error with some better compression
                          pass
                        else:
                          p = i * j;
                          X[i,j] = e;
                      plt.imshow(X)
                      

                      Why did it always missing one subplot when I import mnist digits dataset?

                      copy iconCopydownload iconDownload
                      import keras
                      from keras.datasets import mnist
                      import matplotlib.pyplot as plt
                      (X_train, y_train), (X_test, y_test) = mnist.load_data()
                      fig, axes = plt.subplots(nrows=5, ncols=5, figsize=(8,8))
                      for i, ax in enumerate(axes.flat):
                          ax.imshow(X_train[i], cmap='Greys')
                          ax.set_title("Digit:{}".format(y_train[i]))
                      fig.tight_layout()
                      plt.show()
                      

                      Can not run the the tflite model on Interpreter in android studio

                      copy iconCopydownload iconDownload
                      implementation 'org.tensorflow:tensorflow-lite:0.0.0-nightly-SNAPSHOT'
                      
                      protected MappedByteBuffer loadMappedFile(String filePath) throws IOException {
                          AssetFileDescriptor fileDescriptor = assetManager.openFd(this.directoryName + "/" + filePath);
                      
                          FileInputStream inputStream = new FileInputStream(fileDescriptor.getFileDescriptor());
                          FileChannel fileChannel = inputStream.getChannel();
                          long startOffset = fileDescriptor.getStartOffset();
                          long declaredLength = fileDescriptor.getDeclaredLength();
                          return fileChannel.map(MapMode.READ_ONLY, startOffset, declaredLength);
                        }
                      
                      implementation 'org.tensorflow:tensorflow-lite:0.0.0-nightly-SNAPSHOT'
                      
                      protected MappedByteBuffer loadMappedFile(String filePath) throws IOException {
                          AssetFileDescriptor fileDescriptor = assetManager.openFd(this.directoryName + "/" + filePath);
                      
                          FileInputStream inputStream = new FileInputStream(fileDescriptor.getFileDescriptor());
                          FileChannel fileChannel = inputStream.getChannel();
                          long startOffset = fileDescriptor.getStartOffset();
                          long declaredLength = fileDescriptor.getDeclaredLength();
                          return fileChannel.map(MapMode.READ_ONLY, startOffset, declaredLength);
                        }
                      

                      tensorflow dataset splitting by participant

                      copy iconCopydownload iconDownload
                      def predicate(label, labels_to_filter):
                        return tf.math.reduce_any(tf.equal(label, labels_to_filter))
                      
                      
                      labels_to_filter = tf.constant([0, 1, 2, 3, 4], dtype=tf.int64)
                      subset = dataset.filter(lambda x: predicate(x["participant"], labels_to_filter))
                      

                      Why does my 2-layer neural network achieve just 50% accuracy for binary classification?

                      copy iconCopydownload iconDownload
                      for i in trange(self.n_epoch):
                          for j in range(0, len(x)):
                              a1, a2 = self.forward_propagation(x[i])
                              dw2, db2, dw1, db1 = self.backward_propagation(x[i], y[i], a1, a2)
                              self.update_params(dw2, db2, dw1, db1)
                      
                      def backward_propagation(self, x, y, a1, a2):
                          dz2 = (a2 - y) * (a2*(1-a2))
                          dw2 = np.dot(a1.T, dz2) / len(a1)
                          db2 = dz2/len(dz2)
                          dz1 = np.dot(dz2, self.model_w2.T) * (a1 * (1 - a1))
                          dw1 = np.dot(np.expand_dims(x, 1), dz1) / len(x)
                          db1 = dz1/len(dz1)
                          return dw2, db2, dw1, db1
                      
                      def initialize_params(self, n_0, n_1, n_2):
                          if len(self.model_w1) == 0:
                              self.model_w1 = np.random.randn(n_0, n_1)
                          if len(self.model_w2) == 0:
                              self.model_w2 = np.random.randn(n_1, n_2)
                          if len(self.model_b1) == 0:
                              self.model_b1 = np.random.randn(1, n_1)
                          if len(self.model_b2) == 0:
                              self.model_b2 = np.random.randn(1, n_2)
                      
                      for i in trange(self.n_epoch):
                          for j in range(0, len(x)):
                              a1, a2 = self.forward_propagation(x[i])
                              dw2, db2, dw1, db1 = self.backward_propagation(x[i], y[i], a1, a2)
                              self.update_params(dw2, db2, dw1, db1)
                      
                      def backward_propagation(self, x, y, a1, a2):
                          dz2 = (a2 - y) * (a2*(1-a2))
                          dw2 = np.dot(a1.T, dz2) / len(a1)
                          db2 = dz2/len(dz2)
                          dz1 = np.dot(dz2, self.model_w2.T) * (a1 * (1 - a1))
                          dw1 = np.dot(np.expand_dims(x, 1), dz1) / len(x)
                          db1 = dz1/len(dz1)
                          return dw2, db2, dw1, db1
                      
                      def initialize_params(self, n_0, n_1, n_2):
                          if len(self.model_w1) == 0:
                              self.model_w1 = np.random.randn(n_0, n_1)
                          if len(self.model_w2) == 0:
                              self.model_w2 = np.random.randn(n_1, n_2)
                          if len(self.model_b1) == 0:
                              self.model_b1 = np.random.randn(1, n_1)
                          if len(self.model_b2) == 0:
                              self.model_b2 = np.random.randn(1, n_2)
                      
                      for i in trange(self.n_epoch):
                          for j in range(0, len(x)):
                              a1, a2 = self.forward_propagation(x[i])
                              dw2, db2, dw1, db1 = self.backward_propagation(x[i], y[i], a1, a2)
                              self.update_params(dw2, db2, dw1, db1)
                      
                      def backward_propagation(self, x, y, a1, a2):
                          dz2 = (a2 - y) * (a2*(1-a2))
                          dw2 = np.dot(a1.T, dz2) / len(a1)
                          db2 = dz2/len(dz2)
                          dz1 = np.dot(dz2, self.model_w2.T) * (a1 * (1 - a1))
                          dw1 = np.dot(np.expand_dims(x, 1), dz1) / len(x)
                          db1 = dz1/len(dz1)
                          return dw2, db2, dw1, db1
                      
                      def initialize_params(self, n_0, n_1, n_2):
                          if len(self.model_w1) == 0:
                              self.model_w1 = np.random.randn(n_0, n_1)
                          if len(self.model_w2) == 0:
                              self.model_w2 = np.random.randn(n_1, n_2)
                          if len(self.model_b1) == 0:
                              self.model_b1 = np.random.randn(1, n_1)
                          if len(self.model_b2) == 0:
                              self.model_b2 = np.random.randn(1, n_2)
                      

                      Community Discussions

                      Trending Discussions on MNIST
                      • Audio widget within Jupyter notebook is **not** playing. How can I get the widget to play the audio?
                      • How to calculate maximum gradient for each layer given a mini-batch
                      • Save GAN generated images one by one
                      • Flux.jl : Customizing optimizer
                      • Generator model doesn't produce pictures during training
                      • Use of tf.GradientTape() exhausts all the gpu memory, without it it doesn't matter
                      • partial tucker decomposition
                      • Is it possible to use a collection of hyperspectral 1x1 pixels in a CNN model purposed for more conventional datasets (CIFAR-10/MNIST)?
                      • Why did it always missing one subplot when I import mnist digits dataset?
                      • Can not run the the tflite model on Interpreter in android studio
                      Trending Discussions on MNIST

                      QUESTION

                      Audio widget within Jupyter notebook is **not** playing. How can I get the widget to play the audio?

                      Asked 2022-Mar-15 at 00:07

                      I writing my code within a Jupyter notebook in VS Code. I am hoping to play some of the audio within my data set. However, when I execute the cell, the console reports no errors, produces the widget, but the widget displays 0:00 / 0:00 (see below), indicating there is no sound to play.

                      screen cap of audio widget within .ipynb

                      Below, I have listed two ways to reproduce the error.

                      1. I have acquired data from the hub data store. Looking specifically at the spoken MNIST data set, I cannot get the data from the audio tensor to play
                      import hub
                      from IPython.display import display, Audio
                      from ipywidgets import interactive
                      
                      # Obtain the data using the hub module
                      ds = hub.load("hub://activeloop/spoken_mnist")
                      
                      # Create widget
                      sample = ds.audio[0].numpy()
                      display(Audio(data=sample, rate = 8000, autoplay=True))
                      
                      
                      1. The second example is a test (copied from another post) that I ran to see if it was something wrong with the data or something wrong with my console, environment, etc.
                      # Same imports as shown above
                      
                      # Toy Function to play beats in notebook
                      def beat_freq(f1=220.0, f2=224.0):
                          max_time = 5
                          rate = 8000
                          times = np.linspace(0,max_time,rate*max_time)
                          signal = np.sin(2*np.pi*f1*times) + np.sin(2*np.pi*f2*times)
                          display(Audio(data=signal, rate=rate))
                          return signal
                      
                      v = interactive(beat_freq, f1=(200.0,300.0), f2=(200.0,300.0))
                      display(v)
                      
                      

                      I believe that if it is something wrong with the data (this is a well-known data set so, I doubt it), then only the second one will play. If it is something to do with the IDE or something else, then neither will work, as is the case now.

                      ANSWER

                      Answered 2022-Mar-15 at 00:07

                      Apologies for the late reply! In the future, please tag the questions with activeloop so it's easier to sort through (or hit us up directly in community slack -> slack.activeloop.ai).

                      Regarding the Free Spoken Digit Dataset, I managed to track the error with your usage of activeloop hub and audio display.

                      adding [:,0] to 9th line will help fixing display on Colab as Audio expects one-dimensional data

                      %matplotlib inline
                      import hub
                      from IPython.display import display, Audio
                      from ipywidgets import interactive
                      
                      # Obtain the data using the hub module
                      ds = hub.load("hub://activeloop/spoken_mnist")
                      
                      # Create widget
                      sample = ds.audio[0].numpy()[:,0]
                      display(Audio(data=sample, rate = 8000, autoplay=True))
                      

                      (When we uploaded the dataset, we decided to upload the audio as (N,C) where C is the number of channels, which happens to be 1 for the particular dataset. The added dimension wasn't added automatically)

                      Regarding the VScode... the audio, unfortunately, would still not work (not because of us, but VScode), but you can still try visualizing Free Spoken Digit Dataset (you can play the music there, too). Hopefully this addresses your needs!

                      Let us know if you have further questions.

                      Mikayel from Activeloop

                      Source https://stackoverflow.com/questions/71200390

                      Community Discussions, Code Snippets contain sources that include Stack Exchange Network

                      Vulnerabilities

                      No vulnerabilities reported

                      Install MNIST

                      You can download it from GitHub.
                      You can use MNIST like any standard Java library. Please include the the jar files in your classpath. You can also use any IDE and you can run and debug the MNIST component as you would do with any other Java program. Best practice is to use a build tool that supports dependency management such as Maven or Gradle. For Maven installation, please refer maven.apache.org. For Gradle installation, please refer gradle.org .

                      Support

                      For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .

                      DOWNLOAD this Library from

                      Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from
                      over 430 million Knowledge Items
                      Find more libraries
                      Reuse Solution Kits and Libraries Curated by Popular Use Cases
                      Explore Kits

                      Save this library and start creating your kit

                      Share this Page

                      share link
                      Consider Popular Machine Learning Libraries
                      Try Top Libraries by evolvingstuff
                      Compare Machine Learning Libraries with Highest Support
                      Compare Machine Learning Libraries with Highest Quality
                      Compare Machine Learning Libraries with Highest Security
                      Compare Machine Learning Libraries with Permissive License
                      Compare Machine Learning Libraries with Highest Reuse
                      Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from
                      over 430 million Knowledge Items
                      Find more libraries
                      Reuse Solution Kits and Libraries Curated by Popular Use Cases
                      Explore Kits

                      Save this library and start creating your kit

                      • © 2022 Open Weaver Inc.