FrameCapture | simple frame-by-frame capture tool | Plugin library

You are writing a video buffer to your audioInput and depending on how the buffers arrive, you might also write an audio buffer to your videoInput.

In your case, the CMSampleBuffers contain either audio or video, so you append audio buffers to audioInput and video buffers to videoInput.

You can distinguish the two types of buffer by comparing the output in captureOutput:didOutput: to your audioInput and videoOutput or by looking at the buffer's CMSampleBufferGetFormatDescription()'s CMFormatDescriptionGetMediaType(), but that's more complicated.

You should remove the net.eval() call that comes right after the def infer(net, name):

It needs to be removed because you call this infer function inside your training code. Your model needs to be in train mode throughout the the whole training.

And you never set your model back to train after calling eval as well, so that is the root of the exception you are getting. If you want to use this infer code in your test cases, you can cover that case with an if.

Also the net.eval() that comes right after the total_loss=0 assignment is not useful since you call net.train() right after that. You can also remove that one since it gets neutralized right in next line.

The updated code

Okay, I got it.

See this import from C#.

I think what's happening is that CV2 gives you a binary image that is correctly saved as a frame with 0 under the threshold and 255 (white) above it. Meanwhile, threshold_multiotsu and np.digitize return an image with values 0, 1, 2, all of which look black in the 0-255 range supported by jpeg. You could use skimage.exposure.rescale_intensity to map those values to e.g. 0, 127, 255.

I'm going to guess that the video mainly consist of similar pixels blocked together that the video have compressed to such a low file size. When you load single images into arrays all that compression goes away and depending on the fps of the video you will have thousands of uncompressed images. When you first load an image it will be saved as a numpy array of dtype uint8 and the image size will be WIDTH * HEIGHT * N_COLOR_CHANNELS bytes. After you divide it with 255.0 to normalize between 0 and 1 the dtype changes to float64 and the image size increases eightfold. You can use this information to calculate expected size of the images.

So your options is to either decrease the height and width of your images (downscale), change to grayscale or if your application allows it to stick with uint8 values. If the images doesn't change too much and you don't need thousands of them you could also only save every 10th or whatever seems reasonable. If you need them all as is but they don't fit in memory consider using a generator to load them on demand. It will be slower but at least it will run.

The color property in QML can be fed with QColor or a string (or bound to another property of course) see Qt Docs.

In this case you are feeding it with a property called transparent, however QML cannot find it:

The answer is very simple. You saved your data in a lossy format, namely JPEG, and it lost data.

Use a lossless format like PNG if every bit is important to you.

You should add a metadata output before you'll have some data.