getUserMedia | browser getUserMedia shim with a node.js style error | Runtime Evironment library

In golang I have taken an array ARR1 which represents a time series ( could be audio or in my case an image ) where each element of this time domain array is a floating point value which represents the height of the raw audio curve as it wobbles ... I then fed this floating point array into a FFT call which returned a new array ARR2 by definition in the frequency domain where each element of this array is a single complex number where both the real and the imaginary parts are floating points ... when I then fed this array into an inverse FFT call ( IFFT ) it gave back a floating point array ARR3 in the time domain ... to a first approximation ARR3 matched ARR1 ... needless to say if I then took ARR3 and fed it into a FFT call its output ARR4 would match ARR2 ... essentially you have this time_domain_array --> FFT call -> frequency_domain_array --> InverseFFT call -> time_domain_array ... rinse N repeat

I know Web Audio API has a FFT call ... do not know whether it has an IFFT api call however if no IFFT ( inverse FFT ) you can write your own such function here is how ... iterate across ARR2 and for each element calculate the magnitude of this frequency ( each element of ARR2 represents one frequency and in the literature you will see ARR2 referred to as the frequency bins which simply means each element of the array holds one complex number and as you iterate across the array each successive element represents a distinct frequency starting from element 0 to store frequency 0 and each subsequent array element will represent a frequency defined by adding incr_freq to the frequency of the prior array element )

Each index of ARR2 represents a frequency where element 0 is the DC bias which is the zero offset bias of your input ARR1 curve if its centered about the zero crossing point this value is zero normally element 0 can be ignored ... the difference in frequency between each element of ARR2 is a constant frequency increment which can be calculated using

Note that, at the time I'm writing, this is still experimental territory

You can query permissions using Permissions.query():

You should be using navigator.mediaDevices.getUserMedia. The one in your first example snippet. The second example snippet is deprecated.

The issue here (from what I can see) is that getUserMedia is suppose to request permissions everytime it is called. So the prompt is expected.

You need to store the stream in a variable, and that variable needs to be declared outside of the event handler.

Is there a way to push my local Websocket to a live environment?

It's not straightforward because you need more than vanilla webrtc (which is peer-to-peer). What you want is an SFU. Take a look at mediasoup.

To realize why this is needed think about how the webrtc connection is established in your current app. It's a negotiation between two parties (facilitated by a signaling server). In order to turn this into a multi-cast setup you will need a proxy of sorts that then establishes separate peer-to-peer connections to all senders and receivers.

If everything work on local, and this ice server are set, verify that your gcloud server have the correct firewall for webrtc port (not only your signaling port, check the sdp/ice you exchange). also this Webrtc page allow you to check is a stun/turn work on your client

You will not need stun on your python side, as it's a server his ip may be public (unless you don't want to). Stun allow to find your public ip and allow the port to remain open.

On your server you need to open your signaling port (certainly the WS where you exchange the sdp) and the P2P port (candidate lines in the sdp), the media/data will go through this one. For each media (sdp m line) there are usually one used port.

If any track has not been stopped then your camera will still be active. In your stopStreaming function you only stop the first track in the returned array.

If you instead iterate through the tracks you may catch ones you aren't currently:

You should use the recordTargetStream state as useCallback hook's dependencies.

useCallback will return a memoized version of the callback that only changes if one of the dependencies has changed.

every value referenced inside the callback should also appear in the dependencies array.

index.tsx:

In your drawCanvas function you are setting the canvas width and height to its computed CSS width and height.
When you hide the canvas through display:none, or when it's not appended in the document, these values will be "0px", because the canvas is not in the CSSOM.

A canvas with zero width or height can't produce any output.

Setting the canvas width or height is a very slow operation, in Chrome, this will allocate a full new image bitmap every time you do so, added to the resetting of all the canvas context's default values.
So never set the canvas width or height in all the frames, only when it actually did change, and don't set it to the computed size of your element, set it to the size of your input image, and then set the CSS size accordingly.

After a bunch of trial and error i got to a solution I'm satisfied with. Here is the client side javascript. Server side socket.io server just forwards the data to the correct clients, should be trivial.

There is some frontend stuff in there as well. Just ignore it.

main.js