netron | neural network , deep learning | Machine Learning library

The problem is in the way you specified the shape of accumm_var.

In the input signature you have tf.TensorSpec(shape=None, dtype=tf.float32). Reading the code I see that you are passing a scalar tensor. A scalar tensor is a 0-Dimension tensor, so you should use shape=[] instead of shape=None.

I run here without warnings after annotating extra_function with

The fact that you cannot see Batch Norm when it follows convolution operation has to do with Batch Norm Folding. The convolution operation followed by Batch Norm can be replaced with just one convolution only with different weights.

I assume, for Netron visualization you first convert to ONNX format.

In PyTorch, Batch Norm Folding optimization is performed by default by torch.onnx.export, where eval mode is default one. You can disable this behavior by converting to ONNX in train mode:

torch.onnx.export(..., training=TrainingMode.TRAINING,...)

Then you should be able to see all Batch Norm layers in your graph.

The code now works fine with these changes :

When creating a tensor that will be used in conjuction with a model input feature that is defined with free dimensions (ie: "unk_518"), you need to specify the actual concrete dimension of the tensor.

In your case it looks like you are using SqeezeNet. The first parameter of SqueezeNet corresponds to the batch dimension of the input and so refers to the number of images you wish to bind and run inference on.

Replace the "unk_518" with the batch size that you wish to run inference on:

I fixed it by copying 1 line from the documentation, I needed to add a specific property to the TensorMetadataT object, input_meta.name = "image". This does not come from the model layer name, but needs to be manually added.

This is the correct output for a SSD model. The TFLite_Detection_PostProcess is the custom op that is not run on the EdgeTPU. If you run netron on one of our default SSD models on https://coral.ai/models/, you'll see the PostProcess runs on CPU in that case.

In the case of your model, every part of the of the model has been successfully converted. The last stage (which takes the model output and converts it to various usable outputs) is a custom implementation in TFLite that is already optimized for speed but is generic compute, not TFLite ops that the EdgeTPU accelerates.

As suggested by @JaesungChung, the problem seems to be solved using tf-nightly (I tested on 2.5.0-dev20210325).

It's possible to obtain the same effect in 2.4.0 using a workaround and transforming the Conv1D into Conv2D with a width of 1 and using a flat kernel (1, kernel_size).

Even though there are no dynamic size tensors in the graph, the above graph has a control flow op, While op. Currently, graphs with control flow ops are regarded as dynamic graphs and those graphs are not supported through the hardware acceleration delegates, which allow the only static graph structure.

Should be able to use portpicker.pick_unused_port().

Here's a simple example: https://colab.research.google.com/gist/blois/227d21df87fe8a390c2a23a93b3726f0/netron.ipynb