Introduction
This is an AI vision sensor capable of tracking the head and shoulders of a human body up to a distance of 3 meters, and detecting five types of hand gestures.
In environments with sanitary requirements where contactless operation of equipment is necessary to avoid contamination, this sensor is invaluable. For instance, it can activate disinfection equipment without touch, or power on a range hood even when hands are covered in flour.
In scenarios with high levels of noise, it facilitates contactless remote control. For example, it can adjust the volume of a sound system playing music.
Or perhaps there's a need for equipment to track human movement, discern the presence of a person, and detect their location. For instance, air conditioning or fan devices could follow a person to conserve energy.
This sensor is equipped with the capability to detect five types of hand gestures. It can recognize gestures made at a maximum distance of three meters without contact. By connecting the sensor to your main controller, it can read gesture commands in real time, facilitating the development of devices for contactless operation.
Its head and shoulder tracking feature can control the automatic start and stop of devices. For example, it could be used for community propaganda screens, activating sound or video when a crowd gathers and shutting it off when the crowd disperses to reduce noise pollution.
Moreover, without the need to carry any remote controller, you can silently control the device by simply facing it and making a gesture. This indeed is a remarkable feature.
Features
- An offline AI vision recognition solution that provides accurate and fast identification.
- High sensitivity and high-speed recognition of five types of hand gestures.
- Accurate recognition up to a maximum distance of 3 meters.
- Supports head and shoulder recognition, and can output the number and coordinates of heads and shoulders.
- Two types of data communication: I2C and UART.
- Compatible with 3.3V/5V level and power supply voltage.
- Supports MakeCode and Mind+ graphical programming.
Application
- Contactless control of public area information screens.
- Stage performances.
- Home appliance control.
- Music playback control.
Specifications
Power Parameters
- Supply Voltage: 3.3V-5V
- Level Voltage: 3.3V
- Working Current: 100mA
Interface Parameters
- Communication Interface: PH2.0-4P/2.54mm Pinhole
- Communication Method: I2C/UART
- I2C Address: 0x72
- Default Baud Rate of Serial Port: 9600
- Serial Communication Protocol: Modbus
- Interrupt Interface: 2.54mm Pinhole
Recognition Parameters
- Gesture Recognition Types: 5 Types of Gestures
- Thumbs Up
- Extending the Middle, Ring, and Little Fingers
- Palm Open and Outward
- Extending the Index and Middle Fingers
- Extending the Thumb and Little Finger
- Maximum Number of Faces/Shoulders Recognizable: Up to 10
- Range of Face/Shoulder Recognition: Can recognize when the upper body enters the camera's range
- Face Recognition Score: 0-100
- Face/Shoulder Position Coordinate Recognition: Supported
- Interrupt Output: Outputs low level when any gesture is recognized
- Gesture Recognition Distance: 0.5m-3m
- Face/Shoulder Recognition Distance: 0.5m-3m
- Camera Field of View: D=85°
- Camera Focal Length: 1.56mm
Display Parameters
- Gesture Recognition RGB Light
- Blue Light: Thumbs Up
- Green Light: Extending the Middle, Ring, and Little Fingers
- Red Light: Palm Open and Outward
- Yellow Light: Extending the Index and Middle Fingers
- Purple Light: Extending the Thumb and Little Finger
- Human Detection LED: LED Light (On when people are present, Off when no one is present)
Physical Dimensions
- PCB Size: 42mm*32mm
- Mounting Hole Spacing: 25mm*35mm
- Mounting Hole Diameter: 3.1mm
Function Diagram
Number | Name | Function Description |
---|---|---|
① | D/T | I2C Data Line SDA or UART TXD (3.3V Level) |
② | C/R | I2C Clock Line SCL or UART RXD (3.3V Level) |
③ | GND | Power Negative |
④ | VCC | Power Positive (3.3~5.5V) |
⑤ | INT | Interrupt Output, Outputs Low Level When Any Gesture is Recognized |
⑥ | I2C/UART | I2C/UART Communication Mode Selection Switch |
⑦ | Head-Shoulder Indicator Light | Always on when the camera detects head and shoulders, off when people leave |
⑧ | Gesture Indicator Light | Lights up with the color corresponding to 5 fixed gestures |
Dimension Diagram
Register
Register Type | Device Address (Decimal) |
Modbus Address Offset (Hexadecimal) | Name | R/W | Data Range | Default Value | Description |
---|---|---|---|---|---|---|---|
Input Register | 3001 | 0x0000 | VID | R | 0x0000-0xFFFF | 0x3343 | Vendor ID VID |
Input Register | 3002 | 0x0001 | PID | R | 0x0000-0xFFFF | 0x0272 | Product ID PID |
Input Register | 3003 | 0x0002 | Hardware Version | R | 0x0000-0xFFFF | 0x1000 | Hardware Version: 0x1000 represents V1.0.0.0 |
Input Register | 3004 | 0x0003 | Software Version | R | 0x0000-0xFFFF | 0x1000 | Software Version: 0x1000 represents V1.0.0.0 |
Input Register | 3005 | 0x0004 | Number of Faces | R | 0x0000-0x000A | 0x0000 | Number of Faces Detected by Sensor, Up to 10 |
Input Register | 3006 | 0x0005 | Face x Coordinate | R | 0x0000-0x000A | 0xFFFF | Face x Coordinate Detected by Sensor. 0xFFFF: No Face Detected |
Input Register | 3007 | 0x0006 | Face y Coordinate | R | 0x0000-0x000A | 0xFFFF | Face y Coordinate Detected by Sensor. 0xFFFF: No Face Detected |
Input Register | 3008 | 0x0007 | Face Score | R | 0x0000-0x0064 | 0x0000 | Face Score. 0% ~100% Corresponds to Values 0-100 |
Input Register | 3009 | 0x0008 | Gesture Type | R | 0x0000-0x0005 | 0x0000 | Gesture Type: 0x0000: Other Gestures 0x0001: LIKE 👍 0x0002: OK👌 0x0003: STOP🤚 0x0004: YES ✌️ 0x0005: SIX 🤙 |
Input Register | 3010 | 0x0009 | Gesture Score | R | 0x0000-0x0064 | 0x0000 | Gesture Score. 0% ~100% Corresponds to Values 0-100 |
Holding Register | 4001 | 0x0000 | Device Address | R/W | 0x0001-0x00F7 | 0x0072 | Module's Device Address (1~247) [When the sensor's address is unknown, the broadcast address 0x00 can be used. The sensor will not output data during this operation.] DFRobot assigns different addresses to each product when leaving the factory. Saved after power off, effective immediately |
Holding Register | 4002 | 0x0001 | Serial Parameter Control Register 1 | R/W | 0x0000-0xFFFF | 0x0004 | Module Baud Rate: 0x0001---1200 0x0002---2400 0x0003---4800 0x0004---9600 (Default) 0x0005---14400 0x0006---19200 0x0007---38400 0x0008---57600 0x0009---115200 0x000A---230400 0x000B---460800 0x000C---921600 Other----9600 The module will reply at the current baud rate if the modification is successful, and the new baud rate will be switched after the reply. Saved after power off, effective after restart It is recommended that the device works below 115200 |
Holding Register | 4003 | 0x0002 | Serial Parameter Control Register 2 | R/W | 0x0000-0xFFFF | 0x0003 | High 8-bit Module Parity Bit: 0x00--None (Default) 0x01--Even 0x02--Odd Other--None Low 8-bit Stop Bit: 0x00--0.5 bit 0x01--1 bit 0x02--1.5 bit 0x03--2 bit (Default) Other--2 bit Saved after power off, effective after restart |
Holding Register | 4004 | 0x0003 | Set Target Human Head Shoulder Detection x Coordinate Range Threshold | R/W | 0x0000-0x0064 | 0x0064 | Value Range: 0-100, The Smaller the Value, The Narrower the Detection Range. 100 Represents All Visible Ranges of the Camera |
Holding Register | 4005 | 0x0004 | Set Target Human Head Shoulder Detection Score Threshold | R/W | 0x003C-0x0064 | 0x003C | Value Range: 60-100, Represents 60%-100%. Default Value: 60 |
Holding Register | 4006 | 0x0005 | Set Gesture Detection Score Threshold | R/W | 0x003C-0x0064 | 0x003C | Value Range: 60-100, Represents 60%-100%. Default Value: 60 |
Gesture Type Value Table
Gesture Type | Gesture Indicator Light Color | Program Corresponding Value |
---|---|---|
👍 | Blue | 1 |
👌 | Green | 2 |
🤚 | Red | 3 |
✌️ | Yellow | 4 |
🤙 | Purple | 5 |
Arduino Tutorial
Basic Tutorial
Hardware and Software Preparation
Hardware
- SEN0626 AI Vision Gesture/Face Detection Sensor ×1
- DFR1139 FireBeetle 2 ESP32-E (N16R2) ×1
Software
- Arduino IDE Click to Download Arduino IDE
- Download and Install DFRobot_GestureFaceDetection
Use I2C to Read Data
- Wiring Diagram
- Sample Code
- Connect the module to the ESP32-E controller according to the wiring diagram above, and switch the communication mode selector on the sensor to the I2C side.
- Open Arduino IDE and upload the following code to the ESP32-E controller.
- Open the serial monitor of Arduino IDE, set the baud rate to 115200, and observe the serial print results.
#include "DFRobot_GestureFaceDetection.h"
#define DEVICE_ID 0x72
DFRobot_GestureFaceDetection_I2C gfd(DEVICE_ID);
char str[100];
void setup(){
gfd.begin(&Wire);
Serial.begin(115200);
gfd.setFaceDetectThres(60);
gfd.setGestureDetectThres(60);
gfd.setDetectThres(100);
}
void loop(){
if(gfd.getFaceNumber() > 0){
uint16_t faceScore = gfd.getFaceScore();
uint16_t faceX = gfd.getFaceLocationX();
uint16_t faceY = gfd.getFaceLocationY();
sprintf(str, "detect face at (x = %d, y = %d, score = %d)\n", faceX, faceY, faceScore);
Serial.print(str);
uint16_t gestureType = gfd.getGestureType();
uint16_t gestureScore = gfd.getGestureScore();
sprintf(str, "detect gesture %d, score = %d\n", gestureType, gestureScore);
Serial.print(str);
}
delay(1500);
}
Results
The camera first checks whether a face is recognized. If it is, it retrieves the score and position of the face, formats this information into a string, and outputs it through the serial port. It then retrieves the recognized gesture type and score, again formats this information into a string, and outputs it through the serial port.
The serial monitor will print information similar to the following:
detect face at (x = 123, y = 234, score = 78)
detect gesture 2, score = 88
Where "x=123, y=234" represents the position of the recognized face, "score = 78" represents the score of face recognition. "Gesture 2" represents the recognized gesture type, "score = 88" represents the score of gesture recognition.
Note: The face and gesture need to be recognized by the camera at the same time to output gesture type data.
Use UART to Read Data
- Wiring Diagram
- Sample Code
- Connect the module to the ESP32-E controller according to the wiring diagram above, and switch the communication mode selector on the sensor to the UART side.
- Open Arduino IDE and upload the following code to the ESP32-E controller.
- Open the serial monitor of Arduino IDE, set the baud rate to 115200, and observe the serial print results.
#include "DFRobot_GestureFaceDetection.h"
#define DEVICE_ID 0x72
DFRobot_GestureFaceDetection_UART gfd(&Serial1, DEVICE_ID);
void setup(){
Serial1.begin(9600,SERIAL_8N1, D2, D3);
Serial.begin(115200);
delay(1500);
Serial.print("Product ID: ");
Serial.println(gfd.getPid());
Serial.print("Vendor ID: ");
Serial.println(gfd.getVid());
}
void loop(){
Serial.print("Faces detected: ");
Serial.println(gfd.getFaceNumber());
delay(1500);
}
Results
At initialization, the serial monitor will print out the product ID and vendor ID of the GS01 sensor. Then, in the main loop, it will periodically print out the number of detected human heads and shoulders.
Here is an example of the serial output:
Product ID: 626
Vendor ID: 13123
Faces detected: 1
Where "Product ID: 626" represents the product ID, which is 0x3343 in hexadecimal, "Vendor ID: 13123" represents the vendor ID, which is 0x0272 in hexadecimal, and "Faces detected: 1" indicates that 1 head and shoulder were detected. When the number of detected heads and shoulders is ≥1, the head and shoulder indicator light is always on.
Advanced Tutorial
Colorful Light Ring
Hardware Preparation
- DFR0868 Beetle ESP32-C3 ×1
- FIT0676 93 lights/6 rings WS2812B RGB ×1
- SEN0626 AI Vision Gesture/Face Detection Sensor ×1
Software Preparation
- Arduino IDE Click to Download Arduino IDE
- Download and Install DFRobot_GestureFaceDetection
- Download and Install FastLED
Wiring Diagram
Sample Code
- Connect the AI Vision Gesture/Face Detection Sensor, WS2812B Disc Light, and ESP32-C3 controller according to the wiring diagram above.
- Switch the communication mode selector on the face detection sensor to the I2C side.
- Open Arduino IDE and upload the following code to the ESP32-C3 controller.
#include <Wire.h>
#include <FastLED.h>
#include "DFRobot_GestureFaceDetection.h"
#define LED_PIN 4
#define NUM_LEDS 93
#define DEVICE_ID 0x72
CRGB leds[NUM_LEDS];
DFRobot_GestureFaceDetection_I2C gfd(DEVICE_ID);
void setup() {
FastLED.addLeds<WS2812, LED_PIN, GRB>(leds, NUM_LEDS);
Wire.begin(8, 9); // SDA, SCL
gfd.begin(&Wire);
Serial.begin(115200);
gfd.setFaceDetectThres(60);
gfd.setGestureDetectThres(60);
gfd.setDetectThres(100);
}
void loop() {
if(gfd.getFaceNumber() > 0) {
uint16_t gestureType = gfd.getGestureType();
switch(gestureType) {
case 1:
setAllLEDs(CRGB::Blue);
break;
case 2:
setAllLEDs(CRGB::Green);
break;
case 3:
setAllLEDs(CRGB::Red);
break;
case 4:
setAllLEDs(CRGB::Yellow);
break;
case 5:
setAllLEDs(CRGB::Purple);
break;
default:
setAllLEDs(CRGB::Black);
break;
}
}
else {
setAllLEDs(CRGB::Black);
}
delay(1500);
}
void setAllLEDs(CRGB color) {
for(int i = 0; i < NUM_LEDS; i++) {
leds[i] = color;
}
FastLED.show();
}
Results
Different gestures switch the display color of the light ring. When a person leaves, the light ring turns off.
Human Tracking Fan
Hardware Preparation
- DFR0216 DFRduino UNO R3 ×1
- DFR0265 Gravity: IO Sensor Expansion Board V7.1 ×1
- DFR0332 Gravity: Fan Module ×1
- SER0056 2Kg 300° Clutch Servo ×1
- SEN0626 AI Vision Gesture/Face Detection Sensor ×1
Software Preparation
- Arduino IDE Click to Download Arduino IDE
- Download and Install DFRobot_GestureFaceDetection
Wiring Diagram
Sample Code
- Connect the AI Vision Gesture/Face Detection Sensor to the DFRduino UNO R3 controller according to the wiring diagram above.
- Switch the communication mode selector on the face detection sensor to the I2C side.
- Open Arduino IDE and upload the following code to the DFRduino UNO R3 controller.
#include <Wire.h>
#include <Servo.h>
#include "DFRobot_GestureFaceDetection.h"
#define DEVICE_ID 0x72
#define FAN_PIN 3
#define SERVO_PIN 5
Servo myservo;
DFRobot_GestureFaceDetection_I2C gfd(DEVICE_ID);
void setup(){
myservo.attach(SERVO_PIN);
myservo.write(0);
gfd.begin(&Wire);
Serial.begin(115200);
gfd.setFaceDetectThres(60);
gfd.setGestureDetectThres(60);
gfd.setDetectThres(100);
pinMode(FAN_PIN, OUTPUT);
}
void loop(){
if(gfd.getFaceNumber() > 0){
uint16_t gestureType = gfd.getGestureType();
if(gestureType == 2){
analogWrite(FAN_PIN, 255);
}
else if(gestureType == 3){
analogWrite(FAN_PIN, 0);
}
uint16_t x = gfd.getFaceLocationX();
Serial.println(x);
if(x >= 280 && x <= 400){
myservo.write(45);
delay(2000);
myservo.write(90);
}
else if(x >= 500 && x <= 620){
myservo.write(135);
delay(2000);
myservo.write(90);
}
}
delay(1500);
}
Results
The main goal of this program is to control a servo and a fan based on gesture and face position.
First, the program checks whether a face is detected by the sensor. If a face is detected, the program continues to check the gesture type and the X-coordinate of the face.
For gesture type, if an OK gesture (corresponding value is 2) is detected, the fan will automatically turn on. If a STOP gesture (corresponding value is 3) is detected, the fan will automatically turn off.
For the X-coordinate of the face, if the X-coordinate is in the range of 180-390, the servo will rotate 45° counterclockwise, then pause for 2 seconds and return to the midpoint. If the X-coordinate is in the range of 500-610, the servo will rotate 45° clockwise, then pause for 2 seconds and return to the midpoint.
Note: The X-coordinate should be set according to the actual situation.
API
/**
* @fn getPid
* @brief Get the PID of the device.
* @return PID of the device.
*/
uint16_t getPid();
/**
* @fn getVid
* @brief Get the VID of the device.
* @return VID of the device.
*/
uint16_t getVid();
/**
* @fn setDeviceAddr
* @brief Set the device address.
* @param addr Device address.
* @return True if the address is set successfully, otherwise false.
*/
bool setDeviceAddr(uint16_t addr);
/**
* @fn configUart
* @brief Configure UART settings.
*
* This method is used to set the UART communication parameters for the device, including baud rate, parity, and stop bits.
* Users can choose the appropriate parameters based on their needs to ensure stable and effective communication with the device.
*
* @param baud Baud rate configuration, of type `eBaudConfig_t`, with possible values including:
* - `eBaud_1200` - 1200 baud
* - `eBaud_2400` - 2400 baud
* - `eBaud_4800` - 4800 baud
* - `eBaud_9600` - 9600 baud
* - `eBaud_14400` - 14400 baud
* - `eBaud_19200` - 19200 baud
* - `eBaud_38400` - 38400 baud
* - `eBaud_57600` - 57600 baud
* - `eBaud_115200`- 115200 baud
* - `eBaud_230400`- 230400 baud
* - `eBaud_460800`- 460800 baud
* - `eBaud_921600`- 921600 baud
*
* @param parity Parity configuration, of type `eParityConfig_t`, with possible values including:
* - `UART_CFG_PARITY_NONE` - No parity
* - `UART_CFG_PARITY_ODD` - Odd parity
* - `UART_CFG_PARITY_EVEN` - Even parity
* - `UART_CFG_PARITY_MARK` - Mark parity
* - `UART_CFG_PARITY_SPACE` - Space parity
*
* @param stopBit Stop bits configuration, of type `eStopbits_t`, with possible values including:
* - `UART_CFG_STOP_BITS_0_5` - 0.5 stop bits
* - `UART_CFG_STOP_BITS_1` - 1 stop bit
* - `UART_CFG_STOP_BITS_1_5` - 1.5 stop bits
* - `UART_CFG_STOP_BITS_2` - 2 stop bits
*
* @return Status of the configuration, returning the status code if the configuration is successful; otherwise, it returns an error code.
*/
uint16_t configUart(eBaudConfig_t baud, eParityConfig_t parity, eStopbits_t stopBit);
/**
* @fn setFaceDetectThres
* @brief Set face detection threshold.
*
* Sets the threshold for face detection (0-100). Default is 60%.
*
* @param score Threshold value.
* @return True if successful, otherwise false.
*/
bool setFaceDetectThres(uint16_t score);
/**
* @fn setDetectThres
* @brief Set detection threshold for X coordinate.
*
* Sets the threshold for detecting the X coordinate (0-100). Default is 60%.
*
* @param x Threshold value.
* @return True if successful, otherwise false.
*/
bool setDetectThres(uint16_t x);
/**
* @fn setGestureDetectThres
* @brief Set gesture detection threshold.
*
* Sets the threshold for gesture detection (0-100). Default is 60%.
*
* @param score Threshold value.
* @return True if successful, otherwise false.
*/
bool setGestureDetectThres(uint16_t score);
/**
* @fn getFaceNumber
* @brief Get the number of faces detected by the device.
* @return Number of faces detected.
*/
uint16_t getFaceNumber();
/**
* @fn getFaceLocationX
* @brief Get the X coordinate of the detected face.
* @return X coordinate of the face.
*/
uint16_t getFaceLocationX();
/**
* @fn getFaceLocationY
* @brief Get the Y coordinate of the detected face.
* @return Y coordinate of the face.
*/
uint16_t getFaceLocationY();
/**
* @fn getFaceScore
* @brief Get the score of the detected face.
* @return Score of the face.
*/
uint16_t getFaceScore();
/**
* @fn getGestureType
* @brief Get the type of detected gesture.
*
* This method retrieves the currently detected gesture type. The gesture recognition feature can be used in various applications, such as human-machine interaction or control systems.
* The returned gesture type corresponds to the following values:
* - 1: LIKE (👍) - blue
* - 2: OK (👌) - green
* - 3: STOP (🤚) - red
* - 4: YES (✌️) - yellow
* - 5: SIX (🤙) - purple
*
* If no gesture is detected, the return value may be a specific invalid value (e.g., 0).
*
* @return The detected gesture type, returning the type identifier for the gesture.
*/
uint16_t getGestureType();
/**
* @fn getGestureScore
* @brief Get the score of the detected gesture.
* @return Gesture score.
*/
uint16_t getGestureScore();
Mind+ (Based on Scratch3.0) Graphical Programming
Download and install the Mind+ programming software. Click to Download Mind+
Open Mind+ software, select upload mode, select "Arduino Uno" in "Extension" -> "Main Control Board", then click on the user library.
Copy the following link: https://gitee.com/zhangtang6677/ext-gestureFaceDetectionSensor, paste it into the user library search bar, then search and add the Gesture Face Detection Sensor.
Start programming, the program is as follows:
- Go to the "Connect Device" menu, then click "Upload to Device".
- After the program upload is complete, open the Serial Monitor to see the data output.
Note: In the Mind+ and Makecode environments, only the I2C communication of the gesture sensor can be used, UART communication cannot be used.
Makecode Tutorial
- Open the MakeCode programming platform by entering the following URL in the browser: https://makecode.microbit.org/
- Create a new project and name it.
- Load the UV sensor program library: Click "Settings" -> "Extensions" in turn, then paste the following link in the search box: https://github.com/DFRobot/pxt-DFRobot_GestureFaceDetection
- Start programming, the program is as follows:
- Click download, after the program upload is complete, open "Show Data Device" to see the data output.
Note: In the Mind+ and Makecode environments, only the I2C communication of the gesture sensor can be used, UART communication cannot be used.
More Downloads
Frequently Asked Questions (FAQ)
No customer has any questions about this product yet, feel free to contact us via qq or forum!
For more questions and interesting applications, you can visit the forum for reference or posting.