Semantic-Segmentation | I will upload many semantic segmentation models | Machine Learning library

I am using the Image segmentation guide by fchollet to perform semantic segmentation. I have attempted modifying the guide to suit my dataset by labelling the 8-bit img mask values into 1 and 2 like in the Oxford Pets dataset which will be subtracted to 0 and 1 in class Generator(keras.utils.Sequence).The input image is an RGB-image.

I am not sure why but my dice coefficient isn't increasing at all. I have tried to reduce the learning rate as well as changing the optimizer to SGD/RMSProp, normalizing the data, taking the imbalanced labels into account but the result is very strange. The accuracy/IoU of the model is decreasing as the no. of epochs increases.

If it helps, I previously asked a question about the metrics that I should be using for an imbalanced dataset here. The visualization of the predictions are okay but the metric is not.

What I can do next to debug this problem? Is there anything wrong with my code? Will appreciate any advice.

Here are the results

I'm trying to follow this introduction to how to use VGG-16 to do some Semantic Segmentation. However, there are a few aspect of the tutorial that were unclear to me:

Here is the detailed of their implementation:

We’ll implement FCN-8, as detailed step-by-step below:

• Encoder: A pre-trained VGG16 is used as an encoder. The decoder starts from Layer 7 of VGG16.
• FCN Layer-8: The last fully connected layer of VGG16 is replaced by a 1x1 convolution.

...

What I don't understand is what is layer 7 or 8? Could someone offer me some explanation?

I have a semantic-segmentation model created using Keras.

Now I want to use it in production where I need to execute the model on a large folder with 10k-100k images a few times a day. This takes several hours, so every improvement is helpful.

I'm wondering what is the correct way to use it in production. I currently just use model.predict() on a created Sequence. But everywhere I look I see all kinds of different libraries or technologies that seem relevant.

Tensorflow-serving, converting to C, different libraries by intel and others.

I'm wondering what is the bottom-line recommended way to execute a model as production-grade and as efficiently as possible.

When an image is given as input to a semantic-segmentation model its output is a 2D array of values. Each value in that array represents an object that model thinks is present at that position in the original image. The array looks something like this:

This is the original image:

I am trying to segment 4 lesions with semantic segmentation. I follow this this great post

My training folder has only 2 subfolders with patches: masks and images. Inside the folder with masks, ALL the classes are mixed. The other folder has the corresponding images. So, when I train the model ,it appears: ONE CLASS FOUND, just following the abovementioned post. The results are disappointing and I am wondering if I have to split the classes in the folders, and thus the model recognizes 4 classes instead of the one.

Similar to the issue of The trained model can be deployed on the other platform without dependency of sagemaker or aws service?.

I have trained a model on AWS SageMaker by using the built-in algorithm Semantic Segmentation. This trained model named as model.tar.gz is stored on S3. So I want to download this file from S3 and then use it to make inference on my local PC without using AWS SageMaker anymore. Since the built-in algorithm Semantic Segmentation is built using the MXNet Gluon framework and the Gluon CV toolkit, so I try to refer the documentation of mxnet and gluon-cv to make inference on local PC.

It's easy to download this file from S3, and then I unzip this file to get three files:

1. hyperparams.json: includes the parameters for network architecture, data inputs, and training. Refer to Semantic Segmentation Hyperparameters.
2. model_algo-1
3. model_best.params

Both model_algo-1 and model_best.params are the trained models, and I think it's the output from net.save_parameters (Refer to Train the neural network). I can also load them with the function mxnet.ndarray.load.

Refer to Predict with a pre-trained model. I found there are two necessary things:

1. Reconstruct the network for making inference.
2. Load the trained parameters.

As for reconstructing the network for making inference, since I have used PSPNet from training, so I can use the class gluoncv.model_zoo.PSPNet to reconstruct the network. And I know how to use some services of AWS SageMaker, for example batch transform jobs, to make inference. I want to reproduce it on my local PC. If I use the class gluoncv.model_zoo.PSPNet to reconstruct the network, I can't make sure whether the parameters for this network are same those used on AWS SageMaker while making inference. Because I can't see the image 501404015308.dkr.ecr.ap-northeast-1.amazonaws.com/semantic-segmentation:latest in detail.

As for loading the trained parameters, I can use the load_parameters. But as for model_algo-1 and model_best.params, I don't know which one I should use.

I am trying to use machine learning for semantic segmentation and I managed to find a way to get the proper one hot encoding (using this : https://www.jeremyjordan.me/semantic-segmentation/) however the code that I obtain is quite bad and I am certain that numpy has functionalities that could provide a more elegant solution.

The idea is the following: from a label array (88,240,240) creating a new array (88,240,240,3) with the proper values in each channel.

I came up with this:

Suppose I have one or multiple tiles consisting of a single pattern (e.g. materials like: wood, concrete, gravel...) that I would like to train my classifier on, and then I'll use the trained classifier to determine to which class each pixel in another image belong.

Below are example of two tiles I would like to train the classifier on:

And let's say I want to segment the image below to identify the pixels belonging to the door and those belonging to the wall. It's just an example, I know this image isn't made of exactly the same patterns as the tiles above:

For this specific problem, is it necessary to use convolutional neural networks? Or is there a way to achieve my goal with a shallow neural network or any other classifier, combined with texture features for example?

I've already implemented a classifier with Scikit-learn which works on tile pixels individually (see code below where training_data is a vector of singletons), but I want instead to train the classifier on texture patterns.