SKU SEN0626 Gesture and Face Detection Module Wiki

Introduction

This is an AI vision sensor capable of tracking the head and shoulders of a human body up to a distance of 3 meters, and detecting five types of hand gestures.

In environments with sanitary requirements where contactless operation of equipment is necessary to avoid contamination, this sensor is invaluable. For instance, it can activate disinfection equipment without touch, or power on a range hood even when hands are covered in flour.

In scenarios with high levels of noise, it facilitates contactless remote control. For example, it can adjust the volume of a sound system playing music.

Or perhaps there's a need for equipment to track human movement, discern the presence of a person, and detect their location. For instance, air conditioning or fan devices could follow a person to conserve energy.

This sensor is equipped with the capability to detect five types of hand gestures. It can recognize gestures made at a maximum distance of three meters without contact. By connecting the sensor to your main controller, it can read gesture commands in real time, facilitating the development of devices for contactless operation.

Its head and shoulder tracking feature can control the automatic start and stop of devices. For example, it could be used for community propaganda screens, activating sound or video when a crowd gathers and shutting it off when the crowd disperses to reduce noise pollution.

Moreover, without the need to carry any remote controller, you can silently control the device by simply facing it and making a gesture. This indeed is a remarkable feature.

Features

Application

Specifications

Power Parameters

Interface Parameters

Recognition Parameters

Display Parameters

Physical Dimensions

Function Diagram

Number Name Function Description
① D/T I2C Data Line SDA or UART TXD (3.3V Level)
② C/R I2C Clock Line SCL or UART RXD (3.3V Level)
③ GND Power Negative
④ VCC Power Positive (3.3~5.5V)
⑤ INT Interrupt Output, Outputs Low Level When Any Gesture is Recognized
⑥ I2C/UART I2C/UART Communication Mode Selection Switch
⑦ Head-Shoulder Indicator Light Always on when the camera detects head and shoulders, off when people leave
⑧ Gesture Indicator Light Lights up with the color corresponding to 5 fixed gestures

Dimension Diagram

Register

Register Type Device Address
(Decimal)		 Modbus Address Offset (Hexadecimal) Name R/W Data Range Default Value Description
Input Register 3001 0x0000 VID R 0x0000-0xFFFF 0x0272 Vendor ID VID
Input Register 3002 0x0001 PID R 0x0000-0xFFFF 0x3343 Product ID PID
Input Register 3003 0x0002 Hardware Version R 0x0000-0xFFFF 0x1000 Hardware Version:
0x1000 represents V1.0.0.0
Input Register 3004 0x0003 Software Version R 0x0000-0xFFFF 0x1000 Software Version:
0x1000 represents V1.0.0.0
Input Register 3005 0x0004 Number of Faces R 0x0000-0x000A 0x0000 Number of Faces Detected by Sensor, Up to 10
Input Register 3006 0x0005 Face x Coordinate R 0x0000-0x000A 0xFFFF Face x Coordinate Detected by Sensor.
0xFFFF: No Face Detected
Input Register 3007 0x0006 Face y Coordinate R 0x0000-0x000A 0xFFFF Face y Coordinate Detected by Sensor.
0xFFFF: No Face Detected
Input Register 3008 0x0007 Face Score R 0x0000-0x0064 0x0000 Face Score. 0% ~100% Corresponds to Values 0-100
Input Register 3009 0x0008 Gesture Type R 0x0000-0x0005 0x0000 Gesture Type:
0x0000: Other Gestures
0x0001: LIKE 👍
0x0002: OK👌
0x0003: STOP🤚
0x0004: YES ✌️
0x0005: SIX 🤙
Input Register 3010 0x0009 Gesture Score R 0x0000-0x0064 0x0000 Gesture Score. 0% ~100% Corresponds to Values 0-100
Holding Register 4001 0x0000 Device Address R/W 0x0001-0x00F7 0x0072 Module's Device Address (1~247) [When the sensor's address is unknown, the broadcast address 0x00 can be used. The sensor will not output data during this operation.] DFRobot assigns different addresses to each product when leaving the factory.
Saved after power off, effective immediately
Holding Register 4002 0x0001 Serial Parameter Control Register 1 R/W 0x0000-0xFFFF 0x0004 Module Baud Rate:
0x0001---1200
0x0002---2400
0x0003---4800
0x0004---9600 (Default)
0x0005---14400
0x0006---19200
0x0007---38400
0x0008---57600
0x0009---115200
0x000A---230400
0x000B---460800
0x000C---921600
Other----9600
The module will reply at the current baud rate if the modification is successful, and the new baud rate will be switched after the reply.
Saved after power off, effective after restart
It is recommended that the device works below 115200
Holding Register 4004 0x0003 Set Target Human Head Shoulder Detection x Coordinate Range Threshold R/W 0x0000-0x0064 0x0064 Value Range: 0-100, The Smaller the Value, The Narrower the Detection Range. 100 Represents All Visible Ranges of the Camera
Holding Register 4005 0x0004 Set Target Human Head Shoulder Detection Score Threshold R/W 0x003C-0x0064 0x003C Value Range: 60-100, Represents 60%-100%.
Default Value: 60
Holding Register 4006 0x0005 Set Gesture Detection Score Threshold R/W 0x003C-0x0064 0x003C Value Range: 60-100, Represents 60%-100%.
Default Value: 60

Note: The default device address is 0x0072. In the event of an operation to modify the address, the I2C address will also be updated accordingly.

Commonly Used Function Codes of Modbus-RTU:

Function Code Name Function
0x03 Read Multiple Registers Read integers, characters, status words, floating-point numbers (read N words) --- Read holding registers, byte operations
0x04 Read Input Registers Read integers, status words, floating-point numbers (read N words) --- Read input registers, byte operations
0x06 Write a Single Holding Register Write integers, characters, status words, floating-point numbers (write one word) --- Write holding registers, byte operations
0x10 Write Multiple Holding Registers Write multiple holding registers --- Load specific binary values into a sequence of consecutive holding registers

Through the above commands, input registers and holding registers can be read and written.

Examples of Register Reading and Writing

Hardware Preparation

Example of Reading the Number of Detected Faces

The host reads the number of faces recognized by the sensor through the serial port. The host sends: 72 04 00 04 00 01 7B 08

Device Address------------------------------72
Function Code-------------------------------04//Read Input Registers
Starting Address of the Register------------00 04//Address of the register for reading the number of faces
Number of Registers-------------------------00 01
CRC Check-----------------------------------7B 08

The data of the number of faces returned by the module is: 72 04 02 00 01 7D 3B

Device Address--------------------72
Function Code---------------------04//Read Input Registers
Number of Bytes-------------------00 02//The number of bytes of the returned data
Read Data-------------------------00 01//One face is detected and the number is 1
CRC Check-------------------------7D 3B

Example of Modifying the Baud Rate of the Module

The host sets the baud rate of the sensor to 115200 through the serial port. The host sends: 72 06 00 01 00 09 13 0F

Device Address--------------------------------72
Function Code---------------------------------06//Write Holding Registers
Starting Address of the Register--------------00 01//Address of the baud rate register
Data Written to the Register------------------00 09//The baud rate is 115200
CRC Check-------------------------------------13 0F

The serial port baud rate data returned by the module is: 72 06 00 01 00 09 13 0F

Device Address---------------------------------72
Function Code----------------------------------06//Write Holding Registers
Starting Address of the Register---------------00 01//Address of the baud rate register
Data Written to the Register-------------------00 09//The baud rate is 115200
CRC Check--------------------------------------13 0F

The command sent and the returned value are exactly the same. This indicates that the device has correctly received the sent write request and has returned the correct information.

Note: The settings are saved when the power is off and will take effect after a restart.

Gesture Type Value Table

Gesture Type Gesture Indicator Light Color Program Corresponding Value
👍 Blue 1
👌 Green 2
🤚 Red 3
✌️ Yellow 4
🤙 Purple 5

Arduino Tutorial

Basic Tutorial

Hardware and Software Preparation

Use I2C to Read Data

#include "DFRobot_GestureFaceDetection.h"
#define DEVICE_ID  0x72 

DFRobot_GestureFaceDetection_I2C gfd(DEVICE_ID);
char str[100];

void setup(){
    gfd.begin(&Wire);
    Serial.begin(115200);
    gfd.setFaceDetectThres(60);
    gfd.setGestureDetectThres(60);    
    gfd.setDetectThres(100);
}

void loop(){
    if(gfd.getFaceNumber() > 0){
        uint16_t faceScore = gfd.getFaceScore();
        uint16_t faceX = gfd.getFaceLocationX();
        uint16_t faceY = gfd.getFaceLocationY();
        sprintf(str, "detect face at (x = %d, y = %d, score = %d)\n", faceX, faceY, faceScore);
        Serial.print(str); 
        uint16_t gestureType = gfd.getGestureType();
        uint16_t gestureScore = gfd.getGestureScore();
        sprintf(str, "detect gesture %d, score = %d\n", gestureType, gestureScore);
        Serial.print(str);
    }
    delay(1500);
}

Results

The camera first checks whether a face is recognized. If it is, it retrieves the score and position of the face, formats this information into a string, and outputs it through the serial port. It then retrieves the recognized gesture type and score, again formats this information into a string, and outputs it through the serial port.

The serial monitor will print information similar to the following:

detect face at (x = 123, y = 234, score = 78)
detect gesture 2, score = 88

Where "x=123, y=234" represents the position of the recognized face, "score = 78" represents the score of face recognition. "Gesture 2" represents the recognized gesture type, "score = 88" represents the score of gesture recognition.

Note: The face and gesture need to be recognized by the camera at the same time to output gesture type data.

Use UART to Read Data

#include "DFRobot_GestureFaceDetection.h"
#define DEVICE_ID  0x72 

DFRobot_GestureFaceDetection_UART gfd(&Serial1, DEVICE_ID);

void setup(){
    Serial1.begin(9600,SERIAL_8N1, D2, D3);
    Serial.begin(115200);
    delay(1500);
    Serial.print("Product ID: ");
    Serial.println(gfd.getPid());
    Serial.print("Vendor ID: ");
    Serial.println(gfd.getVid());
}

void loop(){
    Serial.print("Faces detected: ");
    Serial.println(gfd.getFaceNumber());
    delay(1500);
}

Results

At initialization, the serial monitor will print out the product ID and vendor ID of the GS01 sensor. Then, in the main loop, it will periodically print out the number of detected human heads and shoulders.

Here is an example of the serial output:

Product ID: 626
Vendor ID: 13123
Faces detected: 1

Where "Product ID: 626" represents the product ID, which is 0x3343 in hexadecimal, "Vendor ID: 13123" represents the vendor ID, which is 0x0272 in hexadecimal, and "Faces detected: 1" indicates that 1 head and shoulder were detected. When the number of detected heads and shoulders is ≥1, the head and shoulder indicator light is always on.

Advanced Tutorial

Colorful Light Ring

Hardware Preparation

Software Preparation

Wiring Diagram

Sample Code

#include <Wire.h>
#include <FastLED.h>
#include "DFRobot_GestureFaceDetection.h"

#define LED_PIN     4
#define NUM_LEDS    93
#define DEVICE_ID   0x72

CRGB leds[NUM_LEDS];
DFRobot_GestureFaceDetection_I2C gfd(DEVICE_ID);

void setup() {
  FastLED.addLeds<WS2812, LED_PIN, GRB>(leds, NUM_LEDS);
  Wire.begin(8, 9);  // SDA, SCL
  gfd.begin(&Wire);
  Serial.begin(115200);
  gfd.setFaceDetectThres(60);
  gfd.setGestureDetectThres(60);  
  gfd.setDetectThres(100);
}

void loop() {
  if(gfd.getFaceNumber() > 0) {
    uint16_t gestureType = gfd.getGestureType();
    switch(gestureType) {
      case 1:
        setAllLEDs(CRGB::Blue);
        break;
      case 2:
        setAllLEDs(CRGB::Green);
        break;
      case 3:
        setAllLEDs(CRGB::Red);
        break;
      case 4:
        setAllLEDs(CRGB::Yellow);
        break;
      case 5:
        setAllLEDs(CRGB::Purple);
        break;
      default:
        setAllLEDs(CRGB::Black);
        break;
    }
  }
  else {
    setAllLEDs(CRGB::Black);
  }
  delay(1500);
}

void setAllLEDs(CRGB color) {
  for(int i = 0; i < NUM_LEDS; i++) {
    leds[i] = color;
  }
  FastLED.show();
}

Results

Different gestures switch the display color of the light ring. When a person leaves, the light ring turns off.

Human Tracking Fan

Hardware Preparation

Software Preparation

Wiring Diagram

Sample Code

#include <Wire.h>
#include <Servo.h>
#include "DFRobot_GestureFaceDetection.h"

#define DEVICE_ID  0x72
#define FAN_PIN 3
#define SERVO_PIN 5

Servo myservo;
DFRobot_GestureFaceDetection_I2C gfd(DEVICE_ID);

void setup(){
    myservo.attach(SERVO_PIN);
    myservo.write(0);
    gfd.begin(&Wire);
    Serial.begin(115200);
    gfd.setFaceDetectThres(60);
    gfd.setGestureDetectThres(60);
    gfd.setDetectThres(100);
    pinMode(FAN_PIN, OUTPUT);
}

void loop(){
    if(gfd.getFaceNumber() > 0){
        uint16_t gestureType = gfd.getGestureType();
        if(gestureType == 2){ 
            analogWrite(FAN_PIN, 255); 
        }
        else if(gestureType == 3){ 
            analogWrite(FAN_PIN, 0); 
        }

        uint16_t x = gfd.getFaceLocationX();
        Serial.println(x);
        if(x >= 280 && x <= 400){
            myservo.write(45); 
            delay(2000);
            myservo.write(90); 
        }
        else if(x >= 500 && x <= 620){
            myservo.write(135); 
            delay(2000);
            myservo.write(90); 
        }
    }
    delay(1500);
}

Results

The main goal of this program is to control a servo and a fan based on gesture and face position.

First, the program checks whether a face is detected by the sensor. If a face is detected, the program continues to check the gesture type and the X-coordinate of the face.

For gesture type, if an OK gesture (corresponding value is 2) is detected, the fan will automatically turn on. If a STOP gesture (corresponding value is 3) is detected, the fan will automatically turn off.

For the X-coordinate of the face, if the X-coordinate is in the range of 180-390, the servo will rotate 45° counterclockwise, then pause for 2 seconds and return to the midpoint. If the X-coordinate is in the range of 500-610, the servo will rotate 45° clockwise, then pause for 2 seconds and return to the midpoint.

Note: The X-coordinate should be set according to the actual situation.

API

 /**
     * @fn getPid
     * @brief Get the PID of the device.
     * @return PID of the device.
     */
    uint16_t getPid();

    /**
     * @fn getVid
     * @brief Get the VID of the device.
     * @return VID of the device.
     */
    uint16_t getVid();

    /**
     * @fn setDeviceAddr
     * @brief Set the device address.
     * @param addr Device address.
     * @return True if the address is set successfully, otherwise false.
     */
    bool setDeviceAddr(uint16_t addr);

    /**
     * @fn configUart
     * @brief Configure UART settings.
     * 
     * This method is used to set the UART communication parameters for the device, including baud rate, parity, and stop bits. 
     * Users can choose the appropriate parameters based on their needs to ensure stable and effective communication with the device.
     *
     * @param baud Baud rate configuration, of type `eBaudConfig_t`, with possible values including:
     *              - `eBaud_1200`  - 1200 baud
     *              - `eBaud_2400`  - 2400 baud
     *              - `eBaud_4800`  - 4800 baud
     *              - `eBaud_9600`  - 9600 baud
     *              - `eBaud_14400` - 14400 baud
     *              - `eBaud_19200` - 19200 baud
     *              - `eBaud_38400` - 38400 baud
     *              - `eBaud_57600` - 57600 baud
     *              - `eBaud_115200`- 115200 baud
     *              - `eBaud_230400`- 230400 baud
     *              - `eBaud_460800`- 460800 baud
     *              - `eBaud_921600`- 921600 baud
     *
     * @param parity Parity configuration, of type `eParityConfig_t`, with possible values including:
     *              - `UART_CFG_PARITY_NONE`  - No parity
     *              - `UART_CFG_PARITY_ODD`   - Odd parity
     *              - `UART_CFG_PARITY_EVEN`  - Even parity
     *              - `UART_CFG_PARITY_MARK`  - Mark parity
     *              - `UART_CFG_PARITY_SPACE` - Space parity
     *
     * @param stopBit Stop bits configuration, of type `eStopbits_t`, with possible values including:
     *                - `UART_CFG_STOP_BITS_0_5` - 0.5 stop bits
     *                - `UART_CFG_STOP_BITS_1`   - 1 stop bit
     *                - `UART_CFG_STOP_BITS_1_5` - 1.5 stop bits
     *                - `UART_CFG_STOP_BITS_2`   - 2 stop bits
     *
     * @return Status of the configuration, returning the status code if the configuration is successful; otherwise, it returns an error code.
     */
    uint16_t configUart(eBaudConfig_t baud, eParityConfig_t parity, eStopbits_t stopBit);

    /**
     * @fn setFaceDetectThres
     * @brief Set face detection threshold.
     * 
     * Sets the threshold for face detection (0-100). Default is 60%.
     *
     * @param score Threshold value.
     * @return True if successful, otherwise false.
     */
    bool setFaceDetectThres(uint16_t score);

    /**
     * @fn setDetectThres
     * @brief Set detection threshold for X coordinate.
     * 
     * Sets the threshold for detecting the X coordinate (0-100). Default is 60%.
     *
     * @param x Threshold value.
     * @return True if successful, otherwise false.
     */
    bool setDetectThres(uint16_t x);

    /**
     * @fn setGestureDetectThres
     * @brief Set gesture detection threshold.
     * 
     * Sets the threshold for gesture detection (0-100). Default is 60%.
     *
     * @param score Threshold value.
     * @return True if successful, otherwise false.
     */
    bool setGestureDetectThres(uint16_t score);


    /**
     * @fn getFaceNumber
     * @brief Get the number of faces detected by the device.
     * @return Number of faces detected.
     */
    uint16_t getFaceNumber();

    /**
     * @fn getFaceLocationX
     * @brief Get the X coordinate of the detected face.
     * @return X coordinate of the face.
     */
    uint16_t getFaceLocationX();

    /**
     * @fn getFaceLocationY
     * @brief Get the Y coordinate of the detected face.
     * @return Y coordinate of the face.
     */
    uint16_t getFaceLocationY();

    /**
     * @fn getFaceScore
     * @brief Get the score of the detected face.
     * @return Score of the face.
     */
    uint16_t getFaceScore();  

    /**
     * @fn getGestureType
     * @brief Get the type of detected gesture.
     * 
     * This method retrieves the currently detected gesture type. The gesture recognition feature can be used in various applications, such as human-machine interaction or control systems. 
     * The returned gesture type corresponds to the following values:
     * - 1: LIKE (👍) - blue
     * - 2: OK (👌) - green
     * - 3: STOP (🤚) - red
     * - 4: YES (✌️) - yellow
     * - 5: SIX (🤙) - purple
     * 
     * If no gesture is detected, the return value may be a specific invalid value (e.g., 0).
     *
     * @return The detected gesture type, returning the type identifier for the gesture.
     */
    uint16_t getGestureType();

    /**
     * @fn getGestureScore
     * @brief Get the score of the detected gesture.
     * @return Gesture score.
     */
    uint16_t getGestureScore();

Mind+ (Based on Scratch3.0) Graphical Programming

Note: In the Mind+ and Makecode environments, only the I2C communication of the gesture sensor can be used, UART communication cannot be used.

Makecode Tutorial

Note: In the Mind+ and Makecode environments, only the I2C communication of the gesture sensor can be used, UART communication cannot be used.

More Downloads

Frequently Asked Questions (FAQ)

No customer has any questions about this product yet, feel free to contact us via qq or forum!

For more questions and interesting applications, you can visit the forum for reference or posting.