pose_estimation | Webcam Tracking with Tensorflow.js '' By Siraj Raval | Computer Vision library

You have to remap the input topics hector is expecting to the topics you're systems are outputting. Check this page for a full list of topics and params. In the end your launch file should look something like this. Note you need to put in your own topic names.

import numpy as np

For a pose estimation model which outputs a heatmap and offsets. The desired points can be obtained by:

1. Performing a sigmoid operation on the heatmap:

   scores = sigmoid(heatmaps)

2. Each keypoint of those pose is usually represented by a 2-D matrix, the maximum value in that matrix is related to where the model thinks that point is located in the input image. Use argmax2D to obtain the x and y index of that value in each matrix, the value itself represents the confidence value:

   x,y = np.unravel_index(np.argmax(scores[:,:,keypointindex]), scores[:,:,keypointindex].shape) confidences = scores[x,y,keypointindex]

3. That x,y is used to find the corresponding offset vector for calculating the final location of the keypoint:

   offset_vector = (offsets[y,x,keypointindex], offsets[y,x,num_keypoints+keypointindex])

4. After you have obtained your keypoint coords and offsets you can calculate the final position of the keypoint by using ():

   image_positions = np.add(np.array(heatmap_positions) * output_stride, offset_vectors)

See this for determining how to get the output stride, if you don't already have it. The tflite pose estimation has an output stride of 32.

A function which takes output from that Pose Estimation model and outputs keypoints. Not including KeyPoint class

The output stride can be obtained from the following equation:

resolution = ((InputImageSize - 1) / OutputStride) + 1

Example: An input image with a width of 225 pixels and an output stride of 16 results in an output size of 15

15 = ((225 - 1) / 16) + 1

For the tflite PoseNet model:

9 = ((257-1)/ x) + 1 x = 32 so the output stride is 32

Source