Hand Recognition Example with HUSKYLENS 2

HUSKYLENS 2 hand recognition lets Arduino read gesture IDs, counts and positions via I2C and Serial, like giving your project eyes for hand signals. It’s like a camera-based remote control that learns, labels and tracks gestures in real time.

1. Hand Recognition Code Example

Before You Start:

1.1 Output Relevant Data for Hand Recognition

For Hand Recognition, HUSKYLENS 2 can detect hands, draw hand key points, learn gestures, and recognize learned gestures. Relevant data can be printed via the serial port using the program. The data that can be read includes: the hand gesture ID number located near the center of the HUSKYLENS 2 camera screen, the total number of hand gestures detected, and the ID number of the first detected hand gesture.

The example program is as follows.

/*!
 * MindPlus
 * uno
 *
 */
#include "DFRobot_HuskylensV2.h"
// Create object
HuskylensV2 huskylens;


// Main program starts
void setup() {
  Serial.begin(9600);
  Wire.begin();
  while (!huskylens.begin(Wire)) {
    delay(100);
  }
  huskylens.switchAlgorithm(ALGORITHM_HAND_RECOGNITION);
}
void loop() {
  huskylens.getResult(ALGORITHM_HAND_RECOGNITION);
  if ((huskylens.available(ALGORITHM_HAND_RECOGNITION))) {
    Serial.println((String("Gesture ID near the center: ") + String((RET_ITEM_NUM(huskylens.getCachedCenterResult(ALGORITHM_HAND_RECOGNITION), Result, ID)))));
    Serial.println((String("Total number of detected gestures: ") + String((huskylens.getCachedResultNum(ALGORITHM_HAND_RECOGNITION)))));
    Serial.println((String("ID of the 1st detected gesture: ") + String((RET_ITEM_NUM(huskylens.getCachedResultByIndex(ALGORITHM_HAND_RECOGNITION, 1-1), Result, ID)))));
  }
  delay(500);
}

After the program is uploaded, HUSKYLENS 2 enters Hand Recognition mode. Align its camera with the hand gesture on the screen to begin learning. For detailed learning procedures, refer to the Hand Recognition Function.

Open the Serial Monitor to observe the output results.

Running Result: Output the total number of detected hand gestures (any detected gesture, including unlearned ones, will be counted, indicated by a bounding box). Output the ID number of the corresponding gesture as required: learned gestures are assigned IDs in learning order, while unlearned gestures have an ID of 0.

Interface Diagram

1.2 Obtain Relevant Data of Specified Gesture

After HUSKYLENS 2 recognizes a hand gesture, it can retrieve relevant data about the specified gesture in the image. For example, determine whether a learned gesture is present in the image, the name of the specified gesture, and the count of the same gesture detected in the image. When multiple identical gestures appear, you can specify to retrieve parameters of one of them, including name, X/Y coordinates, width, and height.

Example program:

/*!
 * MindPlus
 * uno
 *
 */
#include "DFRobot_HuskylensV2.h"
// Create object
HuskylensV2 huskylens;


// Main program starts
void setup() {
  Serial.begin(9600);
  Wire.begin();
  while (!huskylens.begin(Wire)) {
    delay(100);
  }
  huskylens.switchAlgorithm(ALGORITHM_HAND_RECOGNITION);
}
void loop() {
  huskylens.getResult(ALGORITHM_HAND_RECOGNITION);
  if ((huskylens.available(ALGORITHM_HAND_RECOGNITION))) {
    Serial.println((String("Gesture ID near the center: ") + String((RET_ITEM_NUM(huskylens.getCachedCenterResult(ALGORITHM_HAND_RECOGNITION), Result, ID)))));
    Serial.println((String("Total number of detected gestures: ") + String((huskylens.getCachedResultNum(ALGORITHM_HAND_RECOGNITION)))));
    Serial.println((String("ID of the 1st detected gesture: ") + String((RET_ITEM_NUM(huskylens.getCachedResultByIndex(ALGORITHM_HAND_RECOGNITION, 1-1), Result, ID)))));
  }
  delay(500);
}

Running Result: As shown in the figure, the Running Result displays the number, name, and coordinates of the first detected ID1 gesture in the screen.

Interface Diagram