Instance Segmentation Example for Arduino IDE

Instance segmentation on HuskyLens 2 lets Arduino read object count, IDs and bounding data in real time, like tagging each item in a photo automatically. The code switches to ALGORITHM_SEGMENT, then prints total instances, the instance closest to center, and its ID and center coordinates. A second example shows how to query a specific trained ID, get its name, count all instances of that ID, and read one instance’s X/Y center plus size when multiple similar objects appear.

1. Instance Segmentation Code Example

Before You Start:

1.1 Instance Segmentation and Outputting Relevant Data

Under the Instance Segmentation function, HUSKYLENS can recognize the category of objects in the image and mark the contour of each object. You can use the program to print the total number of instances recognized by HUSKYLENS, the instance near the center (by number), and the name, ID, center point (X and Y) coordinates, as well as the width and height of the specified ID instance.

The example program is as follows.

#include "DFRobot_HuskylensV2.h"
// Create object
HuskylensV2 huskylens;


// Main program starts
void setup() {
	Serial.begin(9600);
	Wire.begin();
  while (!huskylens.begin(Wire)) {
    delay(100);
  }
	huskylens.switchAlgorithm(ALGORITHM_SEGMENT);
}
void loop() {
	huskylens.getResult(ALGORITHM_SEGMENT);
	if ((huskylens.available(ALGORITHM_SEGMENT))) {
		Serial.println((String("Total number of recognized instances: ") + String((huskylens.getCachedResultNum(ALGORITHM_SEGMENT)))));
		Serial.println((String("Center coordinates of the instance near the center: ") + String((String((RET_ITEM_NUM(huskylens.getCachedCenterResult(ALGORITHM_SEGMENT), Result, xCenter))) + String((String(", ") + String((RET_ITEM_NUM(huskylens.getCachedCenterResult(ALGORITHM_SEGMENT), Result, yCenter)))))))));
		Serial.println((String("ID of the 1st recognized instance: ") + String((RET_ITEM_NUM(huskylens.getCachedResultByIndex(ALGORITHM_SEGMENT, 1 - 1), Result, ID)))));
		Serial.println((String("Coordinates of the 1st recognized instance: ") + String((String((RET_ITEM_NUM(huskylens.getCachedResultByIndex(ALGORITHM_SEGMENT, 1 - 1), Result, xCenter))) + String((String(", ") + String((RET_ITEM_NUM(huskylens.getCachedResultByIndex(ALGORITHM_SEGMENT, 1 - 1), Result, yCenter)))))))));
	}
	delay(500);
}

Running Result: After the program is successfully uploaded, HUSKYLENS 2 will automatically switch to the Instance Segmentation function. Align HUSKYLENS 2 with the target object (which must be among the 80 categories), and observe the terminal for data such as the number of recognized instances, the ID of the specified instance, and the center coordinates.

Interface Diagram

1.2 Getting Data of Specified Instances

Under the Instance Segmentation function, after HUSKYLENS 2 is trained, you can obtain data related to specified instances in the image. For example, determine if a trained instance is present in the image, get the name of a specified hand instance, and the quantity of instances of the same category. When multiple instances of the same category appear in the image, you can specify to obtain the parameters of one of the instances, including name, X/Y coordinates, width, and height. You can refer to Instance Segmentation Function Description.

Example program as follows:

#include "DFRobot_HuskylensV2.h"
// Create object
HuskylensV2 huskylens;


// Main program starts
void setup() {
	Serial.begin(9600);
	Wire.begin();
  while (!huskylens.begin(Wire)) {
    delay(100);
  }
	huskylens.switchAlgorithm(ALGORITHM_SEGMENT);
}
void loop() {
	huskylens.getResult(ALGORITHM_SEGMENT);
	if ((huskylens.available(ALGORITHM_SEGMENT))) {
		if (((huskylens.getCachedResultByID(ALGORITHM_SEGMENT, 1) != NULL))) {
			Serial.println((String("Total number of ID1 instances in the frame: ") + String((huskylens.getCachedResultNumByID(ALGORITHM_SEGMENT, 1)))));
			Serial.println((String("Instance name of ID1: ") + String((RET_ITEM_STR(huskylens.getCachedResultByID(ALGORITHM_SEGMENT, 1), Result, name)))));
			Serial.println((String("Coordinates of the 1st detected instance with ID 1: ") + String((String((RET_ITEM_NUM(huskylens.getCachedIndexResultByID(ALGORITHM_SEGMENT, 1, 1-1), Result, xCenter))) + String((String(", ") + String((RET_ITEM_NUM(huskylens.getCachedIndexResultByID(ALGORITHM_SEGMENT, 1, 1-1), Result, xCenter)))))))));
		}
	}
	delay(500);
}

Running Result: After the program is successfully uploaded, HUSKYLENS 2 will automatically switch to the Instance Segmentation function. When multiple instances of the same category appear in the image, it will recognize the count of instances of the same category and output relevant data such as the specified instance's ID, center coordinates, and confidence score.

Interface Diagram