Kinect | Kinect Application Framework including some demo apps

I'm building an application using the webcam to control video games (kinda like a kinect). It uses the webcam (cv2.VideoCapture(0)), AI pose estimation (mediapipe), and custom logic to pipe inputs into dolphin emulator.

The issue is the latency. I've used my phone's hi-speed camera to record myself snapping and found latency of around 32 frames ~133ms between my hand and the frame onscreen. This is before any additional code, just a loop with video read and cv2.imshow (about 15ms)

Is there any way to decrease this latency?

I'm already grabbing the frame in a separate Thread, setting CAP_PROP_BUFFERSIZE to 0, and lowering the CAP_PROP_FRAME_HEIGHT and CAP_PROP_FRAME_WIDTH, but I still get ~133ms of latency. Is there anything else I can be doing?

Here's my code below:

I have a sequences of pngs and corresponding depth files (aligned to the corresponding images) from an external camera.

RGB: 1.png 2.png 3.png etc 150.png

Depth: 1.txt 2.txt 3.txt etc 150.txt

I also have the intrinsics and corresponding camera information in another file called camera.txt.

My goal is to convert these images and depth files to an mkv file in order to utilize the pykinect's body tracker (https://github.com/ibaiGorordo/pyKinectAzure)

So far, I've been able to convert the images and and depth files into an open3D RGBD object. See: http://www.open3d.org/docs/release/python_api/open3d.geometry.RGBDImage.html

I would think we need to run it through the azure kinect reader (https://github.com/isl-org/Open3D/blob/0ec3e5b24551eaffa3c7708aae8630fde9b00e6c/examples/python/reconstruction_system/sensors/azure_kinect_recorder.py#L34), but this seems to open up the camera for additional input.

How can I save this rgbd images to an mkv file format to read in to the pykinect reader? I

In the current commit, ManipulationStation used DepthImageToPointCloud to project the point cloud from color image and depth image input. However, in the documentation, it stated that

Note that if a color image is provided, it must be in the same frame as the depth image.

In my understanding, both color and depth image input come from RgbdSensor which is created from the info from MakeD415CameraModel. Both C and D are two difference frames.

I think the resulting point cloud have a wrong coloring. I tested it on similar setup, but not the MakeD415CameraModel exactly. I currently solved this issue by forcing C and D to be the same frames in MakeD415CameraModel.

My question : Do drake have the method that map between depth image and color image from different frames, similar to kinect library? Since this is the simulation after all, maybe this is overkill?

P.S. I am trying to simulate the image from Azure Kinect; hence the question.

Hello everyone for a university project I am working with the Kinect sensor to get a point cloud. to work with the Kinect I have installed the J4K library for processing, but when I run an example code I get the following message. How can I solve? thank you all.

New to pykinect and kinect in general -- trying to simply get a count of bodies currently being tracked. No skeletal or joint data required. Just want to get a running count of bodies currently in frame. I am using a kinect-v2 and pykinect2.

Being more specific, I'm trying to track how many bodies are in frame and the time elapsed since that value changed. (0 people to 1 person, 1 person to 2, etc.) Due to the built examples for pykinect and the way that they loop, this has proven difficult however. The latest attempt (Now updated with the solved code):

I am trying to calculate distance from a person to the the Kinect sensor v2 in UWP c#.

In WPF, I was getting this by

I am processing frames received from Kinect v2 (Color and IR) in UWP. The program runs on remote machine (XBOX One S). The main goal is to get frames and write them to the disk with 30 fps for Color and IR to later process them further.

I am using the following code to check the frame rate:

I know this sounds stupid and I'm propably very late to the party but here's the thing I want to program an gesture recogniction application (in the likes of this Hand detection or this actual finger detection) for the Xbox 360 Kinect. SDK (version 1.8) is found, installed and works, preliminary research is done - I only forgot to look in which language to write the code. The link from the SDK to the documentation would be the first thing to do but is a dead end, unfortunately.
From the provided examples it seems either to be C++ or C# although some old posts also claim Java. My question is: Is there a documentation not tied to the SDK and which pitfall are there in regard to developing in this specific case under C++/C#/Java? A post from 2011 barely covers the beginning.

Addendum: On further looking I was prompted for the Samples site from the developer toolkit - which can be reached, yet all listed and linked examples are dead ends too.

Addendum: For reference I userd this instruction - ultimately proving futile.

Found an version of NiTE here

I have a dataframe that consists of of video game titles on various platforms. it contains, among other values the name, critic's average score and user's average score. Many of them are missing scores for the user, critic and/or ESRB rating.

What i'd like to do is replace the missing rating, critic and user scores with those for the same game on a different platform (assuming they exist) i'm not quite sure how to approach this.(note - i don't want to drop the duplicate names, because they aren't truly duplicate rows)

here is a sample chunk of the dataframe (i've removed some unrelated columns to make it manageable):