DeepSort | AI powered image tagger backed by DeepDetect | Machine Learning library

I'm currently doing some research to detect and locate a text-cursor (you know, the blinking rectangle shape that indicates the character position when you type on your computer) from a screen-record video. To do that, I've trained YOLOv4 model with custom object dataset (I took a reference from here) and planning to also implement DeepSORT to track the moving cursor.

Here's the example of training data I used to train YOLOv4:

Here's what I want to achieve:

Do you think using YOLOv4 + DeepSORT is considered overkill for this task? I'm asking because as of now, only 70%-80% of the video frame that contains the text-cursor can be successfully detected by the model. If it is overkill after all, do you know any other method that can be implemented for this task?

Anyway, I'm planning to detect the text-cursor not only from Visual Studio Code window, but also from Browser (e.g., Google Chrome) and Text Processor (e.g., Microsoft Word) as well. Something like this:

I'm considering the Sliding Window method as an alternative, but from what I've read, the method might consume much resources and perform slower. I'm also considering Template Matching from OpenCV (like this), but I don't think it will perform better and faster than the YOLOv4.

The constraint is about the performance speed (i.e, how many frames can be processed given amount of time) and the detection accuracy (i.e, I want to avoid letter 'l' or '1' detected as the text-cursor, since those characters are similar in some font). But higher accuracy with slower FPS is acceptable I think.

I'm currently using Python, Tensorflow, and OpenCV for this. Thank you very much!

Okay... So I'm trying to run this repo on my jetson nano 2gb...

I followed this official documentation for installation for tensorflow on jetson nano, while I followed https://www.tensorflow.org/install, this official documentation to run tensorflow on windows..

So after setting the dependencies on windows...the started to run smoothly with no problem.

But after setting the dependencies on jetson nano... the code showed this error..

This is my application architecture:

In my code there is a pedestrian.py file which uses a while loop to read frames from rtsp link and after doing pedestrian detection process (available in this link), it caches the frame in Redis.

(please note that in the loop each time the output frame is replaced with the previous output from loop. it means that there exists only one frame in redis in any moment.)

Then in flask application, I read processed frame from redis and send it for the clients.

This is the code for my pedestrian detection:

I am trying to track objects using the DeepSORT algorithm described in this paper. What I have understood is that, the there are two deep-learning models at work here. One is the object detector (maybe YoLo etc) and the other is a feature extractor. The object detector tries to detect the presence of the object in a frame, while the feature extractor helps to identify if the current detected object has already been detected previously and if so, it assigns the detected object to the corresponding track.

However, one thing I fail to understand is that when does the Object Detector run? Yes, it should run on the first frame, but after that, does it run only after every nth frames? OR does it run on each frame, but only on the apporximate location predicted by the tracker.

Thanks.