Hand Recognition Example

1. Hand Recognition Project Example

Before You Start:

• Overview And Setup
• Hand Recognition Function Guide

1.1 Hand Recognition and Output Related Data

When hand gesture recognition is enabled, hand gestures appearing on the HuskyLens 2 screen will be detected and framed, allowing you to access key data: total detected gestures, total trained gestures, center-closest gesture, nth detected gesture, and specified ID gesture. This data includes the gesture's ID, name, center coordinates (XY), width, and height.

Example program:

Connect to micro:bit and click Download to transfer the program. After successful download, click Show Data to activate data output mode. Align the HuskyLens 2 camera with a hand gesture to observe the output.

As shown, when HuskyLens 2 detects one previously trained gestures named "gesture", the following data will display:

How to learn hand gestures, please refer to the Hand Recognition.

Result: The serial port will output: total detected gestures, ID of the center-closest gesture, name of the first detected gesture, and total the learned gestures.

There are 2 detected gestures, so the dot matrix display shows "2".

1.2 Obtaining Hand Gesture Recognition Feature Data

The system can capture hand gesture feature and positional data for any detected gesture. Key readable information includes: gesture ID, name, width, height, center coordinates (X/Y), wrist coordinates (X/Y), and coordinates for finger bases, joints, and fingertips.

For instance, the sample program below retrieves hand gesture feature data of gestures positioned near the camera's center; this functionality works even for gestures not previously learned.

Result: As shown, click "Download" to transfer the program to micro:bit. After successful download, click "Show Device" to activate data output. Align HuskyLens 2 with a hand gesture, and the serial port will display the gesture ID along with coordinates of its key hand features.

1.3 Obtaining Data of a Specific Hand Gesture

When multiple hand gestures with the same ID appear simultaneously, the program can count their occurrences and retrieve data for the nth gesture in the detected sequence.

Example program:

Result: As shown, click "Download" to transfer the program. Align HuskyLens 2 with two gestures labeled ID 1. After activation, the serial port will output data statistics for these two gestures, including their individual positions and dimensions.

1.4 Hand Gesture Recognition Blocks Description

block	Description
	Used to obtain one set of Hand Gesture Recognition data under the Hand Gesture Recognition function of HUSKYLENS 2.
	Determine if a hand gesture is present in the current frame.
	Retrieves data of the gesture closest to the screen cross cursor, including gesture ID, gesture name, center X/Y coordinates, width, height, and detailed keypoint coordinates including wrist position and finger keypoints.
	Get the total number of gestures detected on the screen, including both unregistered and registered gestures.
	When multiple gestures are detected, retrieves data for a specified gesture.
	Get the total number of learned gestures, regardless of whether they appear in the current frame.
	Determine if a gesture with the specified ID exists in the frame. Unlearned IDs are uniformly set to 0, and learned IDs are sorted in learning order.
	Gets the total number of detected gestures with a specified ID on the screen.
	When multiple gesture are detected, retrieve the relevant data of a specified gesture.
	When multiple gestures with the same ID are detected, retrieve the relevant data of a specified gesture.When multiple gestures with the same ID are detected, retrieve the relevant data of a specified gesture.