SEN0626 Gesture & Face Detection Reference: API, Protocol & Pinout

Library

Arduino IDE Click to download Arduino IDE
Download and install DFRobot_GestureFaceDetection
Download and install DFRobot_RTU

Communication Protocol Description

Communication Methods: I2C/UART
Default Address: 0x72
Default UART Baud Rate: 9600
UART Communication Protocol: modbus

API Description

/**
* @fn getPid
* @brief Retrieves the device's PID.
* @return The device's PID.
*/
uint16_t getPid();

/**
* @fn getVid
* @brief Retrieves the device's VID.
* @return The device's VID.
*/
uint16_t getVid();

/**
* @fn setDeviceAddr
* @brief Sets the device address.
* @param addr The device address.
* @return Returns true if the address is set successfully; otherwise, returns false.
*/
bool setDeviceAddr(uint16_t addr);

/**
* @fn configUart
* @brief Configures UART settings.
* 
* This method is used to set the device's UART communication parameters, including baud rate, parity, and stop bits.
* Users can select appropriate parameters according to their needs to ensure stable and effective communication with the device.
*
* @param baud Baud rate configuration, of type `eBaudConfig_t`, possible values include:
*              - `eBaud_1200`  - 1200 baud
*              - `eBaud_2400`  - 2400 baud
*              - `eBaud_4800`  - 4800 baud
*              - `eBaud_9600`  - 9600 baud
*              - `eBaud_14400` - 14400 baud
*              - `eBaud_19200` - 19200 baud
*              - `eBaud_38400` - 38400 baud
*              - `eBaud_57600` - 57600 baud
*              - `eBaud_115200`- 115200 baud
*              - `eBaud_230400`- 230400 baud
*              - `eBaud_460800`- 460800 baud
*              - `eBaud_921600`- 921600 baud
*
* @param parity Parity configuration, of type `eParityConfig_t`, possible values include:
*              - `UART_CFG_PARITY_NONE`  - No parity
*              - `UART_CFG_PARITY_ODD`   - Odd parity
*              - `UART_CFG_PARITY_EVEN`  - Even parity
*              - `UART_CFG_PARITY_MARK`  - Mark parity
*              - `UART_CFG_PARITY_SPACE` - Space parity
*
* @param stopBit Stop bit configuration, of type `eStopbits_t`, possible values include:
*                - `UART_CFG_STOP_BITS_0_5` - 0.5 stop bits
*                - `UART_CFG_STOP_BITS_1`   - 1 stop bit
*                - `UART_CFG_STOP_BITS_1_5` - 1.5 stop bits
*                - `UART_CFG_STOP_BITS_2`   - 2 stop bits
*
* @return Configuration status, returns a status code if configuration is successful; otherwise, returns an error code.
*/
uint16_t configUart(eBaudConfig_t baud, eParityConfig_t parity, eStopbits_t stopBit);

/**
* @fn setFaceDetectThres
* @brief Sets the face detection threshold.
* 
* Sets the threshold for face detection (0-100). The default value is 60%.
*
* @param score The threshold value.
* @return Returns true if successful; otherwise, returns false.
*/
bool setFaceDetectThres(uint16_t score);

/**
* @fn setDetectThres
* @brief Sets the detection threshold for the X coordinate.
* 
* Sets the threshold for detecting the X coordinate (0-100). The default value is 60%.
*
* @param x The threshold value.
* @return Returns true if successful; otherwise, returns false.
*/
bool setDetectThres(uint16_t x);

/**
* @fn setGestureDetectThres
* @brief Sets the gesture detection threshold.
* 
* Sets the threshold for gesture detection (0-100). The default value is 60%.
*
* @param score The threshold value.
* @return Returns true if successful; otherwise, returns false.
*/
bool setGestureDetectThres(uint16_t score);

/**
* @fn getFaceNumber
* @brief Retrieves the number of faces detected by the device.
* @return The number of detected faces.
*/
uint16_t getFaceNumber();

/**
* @fn getFaceLocationX
* @brief Retrieves the X coordinate of the detected face.
* @return The X coordinate of the face.
*/
uint16_t getFaceLocationX();

/**
* @fn getFaceLocationY
* @brief Retrieves the Y coordinate of the detected face.
* @return The Y coordinate of the face.
*/
uint16_t getFaceLocationY();

/**
* @fn getFaceScore
* @brief Retrieves the score of the detected face.
* @return The face score.
*/
uint16_t getFaceScore();

/**
* @fn getGestureType
* @brief Retrieves the type of detected gesture.
* 
* This method retrieves the currently detected gesture type. The gesture recognition function can be used in various applications such as human-computer interaction or control systems.
* The returned gesture types correspond to the following values:
* - 1: LIKE (?) - Blue
* - 2: OK (?) - Green
* - 3: STOP (?) - Red
* - 4: YES (✌️) - Yellow
* - 5: SIX (?) - Purple
* 
* If no gesture is detected, the return value may be a specific invalid value (e.g., 0).
*
* @return The type of detected gesture.
*/
uint16_t getGestureType();

/**
* @fn getGestureScore
* @brief Retrieves the score of the detected gesture.
* @return The gesture score.
*/
uint16_t getGestureScore();

Detailed Explanation of Core Functions

1. Gesture Recognition Function

Supported Gestures and Corresponding Information

|Action Example| Common Name |Gesture Illustration| Indicator Light Color| Corresponding Program Value
|---|----|---|
|Thumb raised| Good| ?| Blue |1
|Middle, ring, and little fingers extended, with index finger and thumb tip touching (OK gesture) |OK| ?| Green| 2
|Palm spread outward |Stop |? |Red |3
|Index and middle fingers extended| Victory| ✌️| Yellow| 4
|Thumb and little finger extended| Call me| ?| Purple| 5

Core Capabilities

Capable of real-time detection of 5 gestures (?, ?, ?, ✌️, ?) with stable recognition within 3 meters; upon recognition, real-time feedback is provided via corresponding colored indicator lights (e.g., blue for a good), while outputting the gesture type and recognition score.

Frequently Asked Questions

**Q: **
What does the gesture score signify, and what score constitutes a valid recognition?

**A: **
The gesture recognition result outputs a score ranging from 0 to 100. By default, the program a score of ≥60 as a valid gesture (which will trigger the function). A higher score indicates a greater "similarity" between the current action and the standard gesture; for instance, a score of 80 or above generally confirms the target gesture. A score below 30 is often due to suboptimal recognition conditions (such as an unstandardized gesture, excessive distance beyond the effective range, or ambient light interference), in which case accurate gesture recognition is improbable.

**Q: **
What is the range of gesture recognition distance, and what considerations apply when using it at different distances?

A:
• Recognition Range: Supports recognition within 0.5-3 meters. Prior to recognizing gestures, ensure that the area above the shoulders is fully within the camera's field of view and detected (facial indicator light illuminated). If the distance is too close (<0.5 meters), the area above the shoulders may exceed the camera's capture range, resulting in recognition failure;

• Optimal Distance (1-2 meters): Within this range, the camera achieves a balance between "comprehensive frame coverage" and "clarity of feature details," enabling precise capture of gestures with optimal recognition accuracy and response speed;

• Maximum Distance (≤3 meters): More susceptible to environmental interference. Obstructions (such as crowds or furniture) or complex lighting (direct glare/dimness) may impair recognition performance. It is advisable to utilize the 3-meter maximum distance in open, evenly lit environments, prioritizing an unobstructed lens and clear imagery.

2. Facial Detection Function (Including the Area Above the Shoulders)

Core Capabilities

Capable of detecting the "facial range" with feedback via the facial indicator light (lit = presence detected, unlit = no presence).

Capable of detecting the number of faces (up to 10), and real-time output of the number of faces, the coordinates of the face with the highest score, and the corresponding recognition score.

Frequently Asked Questions

**Q: **How to interpret the facial coordinate range, and what is its significance?

**A: **The X/Y coordinate range of the face is 0-640, corresponding to physical position coordinates within the camera's field of view. The numerical value is directly correlated with the actual position: a larger X value indicates the face is further to the right in the camera's view; a larger Y value indicates it is further downward (with the coordinate origin at the top-left corner of the view). For example, (320,320) corresponds to the exact center of the view, facilitating intuitive localization of the face within the frame and enabling position-based functions such as servo tracking.

**Q: **
What does the facial score represent, and what score confirms a face?

A:
The facial recognition result outputs a score ranging from 0 to 100. The program defaults to a reference threshold of 60, with a higher score indicating a greater likelihood that the region is a face. For example, a score of 90 suggests a high probability of it being a face.

Q:
When multiple faces are present simultaneously, what rule governs the return of coordinates?

A:
Currently, simultaneous return of coordinates for multiple faces is not supported; only the coordinates of the face with the highest recognition score (highest score = clearest view, most face-like) will be returned.

Q:
When multiple faces and gestures are detected simultaneously, what is the scoring correspondence rule?

A:
When multiple faces and gestures are detected, the score corresponds by default to the target with the highest gesture score (i.e., the target with the clearest gesture).

Q:
What is the logical sequence of gesture and facial detection, and what precautions apply during use?

A:
The camera first detects whether a face is recognized, then checks for the presence of the 5 gestures. Ensure that both the face and gestures are within the camera's detection range (i.e., perform the corresponding gesture when the facial indicator light is illuminated).

Reference