caffe-c | c to use your caffemodel | Machine Learning library

However I was expecting a single value per channel instead I found a 256x256 array: does it mean that the took a mean on each pixel of each channel?

Exactly. According to the shape of mean.binaryproto, this file is the average image of some dataset, which means that it took the mean of each pixel (feature) for each channel.

This should not be confused with the mean pixel, which, as you stated, is a single value for each channel.

For example, mean pixel was adoped by Very Deep Convolutional Networks for Large-Scale Image Recognition. According to their paper:

The only pre-processing we do is subtracting the mean RGB value, computed on the training set, from each pixel

In other words, if you consider an RGB image to be 3 feature arrays of size N x N, the average image will be the mean of each feature and the mean pixel will be the mean of all features.

Another question is the following: I want to use such NN with OpenCV which instead of RGB uses BGR: How to know if the mean 3x256x256 uses RGB or BGR?

I doubt the binary file you are reading stores any information about its color format, but a practical way to figure out is to plot this image using matplotlib and see if the colors make sense.

For example, face images. If red and blue channels are swapped the skin tone will look blueish.

In fact, the image above is an example of average image (face images) :)

You could also assume it is BGR since OpenCV uses this color format.

However, the correct way to find out how this mean.binaryproto was generated is by looking at their repositories or by asking the owner of the model.

Problem solved: The error came up because caffe build wasn't done successfully. I recommend not to go up with the sudo apt install caffe-cpu command (Which is mentioned in the official caffe installation guide for Ubuntu); because it will end up in the error as above. It's better to install from the source.

Let me give step by step guidance to install caffe successfully in Ubuntu 18.04 LTS:

1] sudo apt-get install -y --no-install-recommends libboost-all-dev

2] sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libboost-all-dev libhdf5-serial-dev \ libgflags-dev libgoogle-glog-dev liblmdb-dev protobuf-compiler

3] git clone https://github.com/BVLC/caffe cd caffe cp Makefile.config.example Makefile.config

4] sudo pip install scikit-image protobuf cd python for req in $(cat requirements.txt); do sudo pip install $req; done

5] Modify the Makefile.config file: Uncomment the line CPU_ONLY := 1, and the line OPENCV_VERSION := 3.

6] Find LIBRARIES line in Makefile and change it to as follows:

LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_hl hdf5 \ opencv_core opencv_highgui opencv_imgproc opencv_imgcodecs

7] make all

Now you could get some error like this:

CXX src/caffe/net.cpp src/caffe/net.cpp:8:18: fatal error: hdf5.h: No such file or directory compilation terminated. Makefile:575: recipe for target '.build_release/src/caffe/net.o' failed make: *** [.build_release/src/caffe/net.o] Error 1

To solve this error follow step 8.

8] install libhdf5-dev open Makefile.config, locate line containing LIBRARY_DIRS and append /usr/lib /x86_64-linux-gnu/hdf5/serial locate INCLUDE_DIRS and append /usr/include/hdf5/serial/ (per this SO answer) rerun make all

9] make test

10] make runtest

11] make pycaffe

Now you could get some error like this:

CXX/LD -o python/caffe/_caffe.so python/caffe/_caffe.cpp python/caffe/_caffe.cpp:10:31: fatal error: numpy/arrayobject.h: No such file or directory compilation terminated. Makefile:501: recipe for target 'python/caffe/_caffe.so' failed make: *** [python/caffe/_caffe.so] Error 1

To solve this error follow step 12.

12] Find PYTHON_INCLUDE line in Makefile.config and do the changes as follows:

I was able to get it working following these steps,

1. Get caffe source form here (https://github.com/BVLC/caffe.git)

2. Install CUDA if you need GPS support (https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1804)

3. Install CUDNN, if you need GPS support (https://developer.nvidia.com/rdp/cudnn-download)

4. Replace existing Makefile.config with this (https://gist.github.com/GPrathap/1f9d184c55779509860b8bf92cea416d) Here I have configured for cuda 9.2. If you have a different version please search 9.2 and change the version which you have instaled. Also, please recheck all the paths which are declared in Makefile.config.

5. You may type make all followed by make test

6. make distribute for creating the final lib, an include directory of caffe which can be found in (caffe/distribute).

7. If you are using CMake type project add where to find caffe as bellow,

When you install caffe on ubuntu using sudo apt install caffe-cpu, it compiles the bindings for python 3 only (_caffe.cpython-36m-x86_64-linux-gnu.so), which is located at /usr/lib/python3/dist-packages/caffe/. So the short answer is to use python 3 instead.

The long answer is to compile caffe with python 2 bindings from source.

You are completely right and well done for finding this. Given the similarity of code, my guess is there is one source bug, and then papers after papers copied the bad implementation without examining it closely.

The "akturtle" issue raiser is completely right too, I was going to give the same example. I'm not sure if "kunhe" understood the argument, of course recall matters when computing average precision.

Yes, the bug should inflate the numbers. I just hope that the ranking lists are long enough and that the methods are reasonable enough such that they achieve 100% recall in the ranked list, in which case the bug would not affect the results.

Unfortunately it's hard for reviewers to catch this as typically one doesn't review code of papers.. It's worth contacting authors to try to make them update the code, update their papers with correct numbers, or at least don't continue making the mistake in their future works. If you are planning to write a paper comparing different methods, you could point out the problem and report the correct numbers (as well as potentially the ones with the bug just to make apples for apples comparisons).

To answer your side-question:

Is MAP@k always less than MAP computed for all ranked list?

Not necessarily, MAP@k is essentially computing the MAP while normalizing for the potential case where you can't do any better given just k retrievals. E.g. consider returned ranked list with relevances: 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 and assume there are in total 6 relevant documents. MAP should be slightly higher than 50% here, while MAP@3 = 100% because you can't do any better than retrieving 1 1 1. But this is unrelated to the bug you discovered as with their bug the MAP@k is guaranteed to be at least as large as the true MAP@k.

I tried on my end to install caffe-cpu, and it seems that the file /etc/apt/sources.list doesn't have the needed repositories to install it, in the datalab instance. To workaround this issue, I used the following commands, in a created notebook:

I have fixed this problem eventually. I am not 100% sure what worked but it was most likely changing the bias to 0 while learning.

If anyone has been wondering, it turns out I have trained the model with a different version of caffe and was trying to test with another. I have two versions installed on my computer and it seems I was simply importing the older one during testing with python script (for training I had directly referenced and used the caffe tools under build) that is defined in LD_LIBRARY_PATH. The difference between versions is not too dramatic, but it seems there was a mismatch while reading prototoxt.