Hand Tracking CH-1

First of all, What is MediaPipe?

“MediaPipe is a framework for building multimodal (eg. video, audio, any time series data), cross platform (i.e Android, iOS, web, edge devices) applied ML pipelines. With MediaPipe, a perception pipeline can be built as a graph of modular components, including, for instance, inference models (e.g., TensorFlow, TFLite) and media processing functions.”

*Face Detection

ML solution example with MediaPipe:

*Face Detection

*Multi-hand Tracking

*Hair Segmentation

*Object Detection and Tracking

Object Detection and Tracking

*Objectron: 3D Object Detection and Tracking

*AutoFlip: Automatic video cropping pipeline

Hand Landmark Model

They manually annotated 30 000 images of different hands. Within just two clicks we are going to run.

1. We need to install mediapipe and opencv packages.

2. Importing Libraries

3. Try to open our camera

This line can be : cap = cv2.VideoCapture(0) or cap = cv2.VideoCapture(1)

MediaPipe works with RGB, so we have to convert the color BGR to RGB.

….

Now, we can see the frame.

But, there are no connections. We will add “HAND_CONNECTIONS” code.

It’s good to see that but we can not track POSITIONS.

Also, before that let’s find frames per second-fps.

Now, we will get id number each of hand (with the handLm part) and landmark information that will show the x and y coordinates to us.

Now, we can see the positions when we run the code.

For ex:

17 x: 0.7893819212913513
y: 0.8477940559387207
z: -0.10227853804826736

18 x: 0.7618409991264343
y: 0.8384567499160767
z: -0.13025076687335968

After that, we will see how to get pixels?

…..

id| w | h

15 448 556
16 451 570
17 473 500
18 462 537
19 460 554

Circle Point

So,the last part to detect id=0 , we will add this code:

(You can also try id=4, it will show the thumb. If you don’t write id number, it will show all the points on your hand.)

Now, we can create a module file with these code, and we can use it later.

Language: Python

IDE: VS Code

OS: Linux

Source:

1. https://opensource.google/projects/mediapipe
2. Murtaza’s Workshop YouTube
3. https://github.com/zehranrgi/ComputerVision/tree/main/handtracking