pose_estimation | Pose estimation of a 2D picture , given a 3D bundler output

import numpy as np

For a pose estimation model which outputs a heatmap and offsets. The desired points can be obtained by:

1. Performing a sigmoid operation on the heatmap:

   scores = sigmoid(heatmaps)

2. Each keypoint of those pose is usually represented by a 2-D matrix, the maximum value in that matrix is related to where the model thinks that point is located in the input image. Use argmax2D to obtain the x and y index of that value in each matrix, the value itself represents the confidence value:

   x,y = np.unravel_index(np.argmax(scores[:,:,keypointindex]), scores[:,:,keypointindex].shape) confidences = scores[x,y,keypointindex]

3. That x,y is used to find the corresponding offset vector for calculating the final location of the keypoint:

   offset_vector = (offsets[y,x,keypointindex], offsets[y,x,num_keypoints+keypointindex])

4. After you have obtained your keypoint coords and offsets you can calculate the final position of the keypoint by using ():

   image_positions = np.add(np.array(heatmap_positions) * output_stride, offset_vectors)

See this for determining how to get the output stride, if you don't already have it. The tflite pose estimation has an output stride of 32.

A function which takes output from that Pose Estimation model and outputs keypoints. Not including KeyPoint class