Hand Recognition Example

1. Hand Recognition Example Program

Before You Start:

• Overview And Setup
• Hand Recognition Function Guide

1.1 Output Data for Hand Recognition

The HUSKYLENS 2 can detect hands within its field of view and obtain hand recognition data. The readable data includes: total number of detected hands in the image, ID of the hand closest to the center of the HUSKYLENS 2 view, and ID of the first detected hand.

The example program is as follows.

#include "unihiker_k10.h"
#include "DFRobot_HuskylensV2.h"
// Create objects (Huskylens: hand recognition; UNIHIKER K10: screen)
HuskylensV2  huskylens;
UNIHIKER_K10 k10;
uint8_t      screen_dir = 2;  // Screen display direction


// Main program initialization
void setup() {
  k10.begin();          // Initialize UNIHIKER K10
  Wire.begin();         // Initialize I2C communication

  // Wait for Huskylens connection (retry every 100ms)
  while (!huskylens.begin(Wire)) {
    delay(100);
  }

  k10.initScreen(screen_dir);       // Init screen with set direction
  k10.creatCanvas();                // Create drawing canvas
  huskylens.switchAlgorithm(ALGORITHM_HAND_RECOGNITION);  // Set to hand recognition mode
}

void loop() {
  huskylens.getResult(ALGORITHM_HAND_RECOGNITION);  // Get latest hand recognition data

  // Check if hand recognition data is available
  if (huskylens.available(ALGORITHM_HAND_RECOGNITION)) {
    // Simplify index: 1-1 → 0 (more intuitive, same meaning)
    k10.canvas->canvasText(String("Total hands: ") + String(huskylens.getCachedResultNum(ALGORITHM_HAND_RECOGNITION)), 1, 0xFF0000);
    k10.canvas->canvasText(String("Center hand ID: ") + String(RET_ITEM_NUM(huskylens.getCachedCenterResult(ALGORITHM_HAND_RECOGNITION), Result, ID)), 3, 0xFF0000);
    k10.canvas->canvasText(String("First hand ID: ") + String(RET_ITEM_NUM(huskylens.getCachedResultByIndex(ALGORITHM_HAND_RECOGNITION, 0), Result, ID)), 5, 0xFF0000);
    
    k10.canvas->updateCanvas();  // Refresh screen
    delay(50);
  }
}

After uploading the program, wait for the upload to complete.

HuskyLens 2 enters the Hand Recognition function. Align HuskyLens 2's camera with the gesture in the frame for learning. For detailed steps on how to learn gestures, please refer to: Hand Recognition Function.

Running result:The total number of detected gestures can be output. Regardless of whether they are learned gestures or not, all detected gestures (enclosed in a bounding box) will be counted. The corresponding gesture's ID number can be output as required. Learned gestures will be assigned ID numbers in the order of learning, and unlearned gestures will have an ID number of 0.

1.2 Get Hand Gesture Key Point Data

Retrieve ID, name, and key point data for specified hand gestures, including: Gesture ID, Gesture Name, X and Y coordinates of the gesture center point, width, height, wrist X/Y coordinates, and X/Y coordinates of the root, joint, and tip of each finger. Refer to the Hand Recognition blocks documentation for detailed data specifications.

The following example program retrieves X/Y coordinates of the wrist and fingertips for hand gestures positioned near the center of the camera view, including unlearned gestures.

#include "unihiker_k10.h"
#include "DFRobot_HuskylensV2.h"
// Create objects (Huskylens for hand recognition, UNIHIKER K10 for screen)
HuskylensV2  huskylens;
UNIHIKER_K10 k10;
uint8_t      screen_dir = 2;  // Screen display direction


// Main program initialization
void setup() {
  k10.begin();          // Initialize UNIHIKER K10 hardware
  Wire.begin();         // Initialize I2C communication for Huskylens

  // Wait for Huskylens connection (retry every 100ms until success)
  while (!huskylens.begin(Wire)) {
    delay(100);
  }

  k10.initScreen(screen_dir);       // Initialize screen with set direction
  k10.creatCanvas();                // Create canvas for screen display
  huskylens.switchAlgorithm(ALGORITHM_HAND_RECOGNITION);  // Set Huskylens to hand recognition mode
}

void loop() {
  huskylens.getResult(ALGORITHM_HAND_RECOGNITION);  // Fetch latest hand recognition data

  // Check if valid hand recognition data is available
  if (huskylens.available(ALGORITHM_HAND_RECOGNITION)) {
    // Store center hand's data to avoid repeated function calls (improve efficiency)
    auto centerHand = huskylens.getCachedCenterResult(ALGORITHM_HAND_RECOGNITION);
    
    // Display hand info (red color: 0xFF0000)
    k10.canvas->canvasText(String("Center hand ID: ") + String(RET_ITEM_NUM(centerHand, Result, ID)), 1, 0xFF0000);
    k10.canvas->canvasText(String("Wrist: ") + String(RET_ITEM_NUM(centerHand, HandResult, wrist_x)) + String(",") + String(RET_ITEM_NUM(centerHand, HandResult, wrist_y)), 3, 0xFF0000);
    k10.canvas->canvasText(String("Thumb tip: ") + String(RET_ITEM_NUM(centerHand, HandResult, thumb_tip_x)) + String(",") + String(RET_ITEM_NUM(centerHand, HandResult, thumb_tip_y)), 5, 0xFF0000);
    k10.canvas->canvasText(String("Index tip: ") + String(RET_ITEM_NUM(centerHand, HandResult, index_finger_tip_x)) + String(",") + String(RET_ITEM_NUM(centerHand, HandResult, index_finger_tip_y)), 6, 0xFF0000);
    k10.canvas->canvasText(String("Middle tip: ") + String(RET_ITEM_NUM(centerHand, HandResult, middle_finger_tip_x)) + String(",") + String(RET_ITEM_NUM(centerHand, HandResult, middle_finger_tip_y)), 7, 0xFF0000);
    k10.canvas->canvasText(String("Ring tip: ") + String(RET_ITEM_NUM(centerHand, HandResult, ring_finger_tip_x)) + String(",") + String(RET_ITEM_NUM(centerHand, HandResult, ring_finger_tip_y)), 8, 0xFF0000);
    k10.canvas->canvasText(String("Pinky tip: ") + String(RET_ITEM_NUM(centerHand, HandResult, pinky_finger_tip_x)) + String(",") + String(RET_ITEM_NUM(centerHand, HandResult, pinky_finger_tip_y)), 9, 0xFF0000);
    
    k10.canvas->updateCanvas();  // Refresh screen to show latest data
  }
}

Running Result: As shown in the figure, after running the program, the UNIHIKER K10 screen displays the gesture ID and the key point data of this gesture. Since the gesture has been learned first, so the gesture ID is 1.

1.3 Obtaining Relevant Data of Specified Gestures

In addition to the above data, more specified hand gesture data can be obtained. For example, determining if a specified hand gesture appears in the frame, getting the name of the specified gesture, and counting the number of identical gestures in the frame. When multiple identical hand gestures appear, you can specify which one to obtain the relevant parameters of, including name, center X/Y coordinates, width, height, fingertip coordinates, wrist coordinates, etc.

Example program:

#include "unihiker_k10.h"
#include "DFRobot_HuskylensV2.h"
// Create objects (Huskylens: hand recognition; UNIHIKER K10: screen)
HuskylensV2  huskylens;
UNIHIKER_K10 k10;
uint8_t      screen_dir = 2;  // Screen display direction


// Main program initialization
void setup() {
  k10.begin();          // Initialize UNIHIKER K10
  Wire.begin();         // Initialize I2C communication

  // Wait for Huskylens connection (retry every 100ms)
  while (!huskylens.begin(Wire)) {
    delay(100);
  }

  k10.initScreen(screen_dir);       // Init screen with set direction
  k10.creatCanvas();                // Create drawing canvas
  huskylens.switchAlgorithm(ALGORITHM_HAND_RECOGNITION);  // Set to hand recognition mode
}

void loop() {
  huskylens.getResult(ALGORITHM_HAND_RECOGNITION);  // Get latest hand recognition data

  if (huskylens.available(ALGORITHM_HAND_RECOGNITION)) {
    // Display center hand ID
    k10.canvas->canvasText(String("Center hand ID: ") + String(RET_ITEM_NUM(huskylens.getCachedCenterResult(ALGORITHM_HAND_RECOGNITION), Result, ID)), 1, 0xFF0000);

    // Check if ID1 hand exists and display its info
    if (huskylens.getCachedResultByID(ALGORITHM_HAND_RECOGNITION, 1) != NULL) {
      // Simplify index: 1-1 → 0 (more intuitive)
      k10.canvas->canvasText(String("ID1 hand count: ") + String(huskylens.getCachedResultNumByID(ALGORITHM_HAND_RECOGNITION, 1)), 1, 0xFF0000);
      k10.canvas->canvasText(String("ID1 hand name: ") , 3, 0xFF0000);
      k10.canvas->canvasText(String(RET_ITEM_STR(huskylens.getCachedResultByID(ALGORITHM_HAND_RECOGNITION, 1), Result, name)), 4, 0xFF0000);
      k10.canvas->canvasText("First ID1 hand", 5, 0xFF0000);
      k10.canvas->canvasText(String("Coords: ") + String(RET_ITEM_NUM(huskylens.getCachedIndexResultByID(ALGORITHM_HAND_RECOGNITION, 1, 0), Result, xCenter)) + String(",") + String(RET_ITEM_NUM(huskylens.getCachedIndexResultByID(ALGORITHM_HAND_RECOGNITION, 1, 0), Result, yCenter)), 6, 0xFF0000);
      
      k10.canvas->updateCanvas();  // Refresh screen
    }
  }
}

Running result: As shown in the figure, you can obtain the count and name of ID1 gestures in the image, as well as the coordinates of the first detected ID1 gesture.