VideoFrame | HTML5 Video SMPTE Time Code and Frame Seeking API | Video Utils library

Even though not directly an OpenCV API solution, converting the byte array to short array in Java seems to work:

Here you have two examples. In both examples I added the SVG namespace to the string, be cause it is a separate XML/SVG document and not haft of the HTML.

In the first example I just create a data URL and insert that as the source of an image object. Here you need to set the width and the height.

In the second example I used Blob and the function URL.createObjectURL().

I have found the solution. This is the new main loop:

Disclaimer:
I am not an expert in this field and it's my first use of this API in this way. The specs and the current implementation don't seem to match, and it's very likely that things will change in the near future. So take this answer with all the salt you can, it is only backed by trial.

There are a few things that seems wrong in your implementation:

• duration and timestamp are set in micro-seconds, that's 1/1,000,000s. Your 500 duration is then only half a millisecond, that would be something like 2000FPS and your three images would get all displayed in 1.5ms. You will want to change that.
• In current Chrome's implementation, you need to specify the displayWidth and displayHeight members of the VideoFrameInit dictionary (though if I read the specs correctly that should have defaulted to the source image's width and height).

Then there is something I'm less sure about, but it seems that you can't batch-write many frames. It seems that the timestamp field is kind of useless in this case (even though it's required to be there, even with nonsensical values). Once again, specs have changed so it's hard to know if it's an implementation bug, or if it's supposed to work like that, but anyway it is how it is (unless I too missed something).

So to workaround that limitation you'll need to write periodically to the stream and append the frames when you want them to appear.

Here is one example of this, trying to keep it close to your own implementation by writing a new frame to the WritableStream when we want it to be presented.

I'm not sure what the issue is with the original sandbox or with the adaptation to react-easy-crop but I was able to easily adapt it to the desired library (despite not liking its UI very much, but to each his own, I guess)

In any case, here's a working sandbox with react-easy-crop: https://codesandbox.io/s/react-uploady-crop-and-upload-with-react-easy-crop-5g7vw

Including here the preview item that I updated:

The video has frame rate of 25fps, and not 24fps:

After putting the correct value it works ok: demo
The VideoFrame api heavily relies on FPS provided by you. You can find FPS of your videos offline and send as metadata along with stop frames from server.

The site videoplayer.handmadeproductions.de uses window.requestAnimationFrame() to get the callback.

There is a new better alternative to requestAnimationFrame. The requestVideoFrameCallback(), allows us to do per-video-frame operations on video.
The same functionality, you domed in OP, can be achieved like this:

VideoEncoder and other classes from the WebCodecs API provide you with the way of encoding your images as frames in a video stream, however encoding is just the first step in creating a playable multimedia file. A file like this may potentially contain multiple streams - for instance when you have a video with sound, that's already at least one video and one audio stream, so a total of two. You need additional container format to store the streams so that you do not have to send the streams in separate files. To create a container file from any number of streams (even just one) you need a multiplexer (muxer for short). Good summary of the topic can be found in this Stack Overflow answer, but to quote the important part:

1. When you create a multimedia file, you use a coder algorithms to encode the video and audio data, then you use a muxer to put the streams together into a file (container). To play the file, a demuxer takes apart the streams and feeds them into decoders to obtain the video and audio data.
2. Codec means coder/decoder, and is a separate concept from the container format. Many container formats can hold lots of different types of format (AVI and QuickTime/MOV are very general). Other formats are restricted to one or two media types.

You may think "i have only one stream, do i really need a container?" but multimedia players expect received data (either data read from a file or streamed over network) to be in a container format. Even if you have only one video stream, you still need to pack it into a container for them to recognize it.

Joining the byte buffers into one big blob of data will not work:

Since you are using the new Image() function and generating a HTMLImageElement object dynamically, add img.src after the onload function like so:

Found the answer here.

If Windows ML does not support your model's color format or pixel range, then you can implement conversions and tensorization. You'll create an NCHW four-dimensional tensor for 32-bit floats for your input value. See the Custom Tensorization Sample for an example of how to do this.