HUSKYLENS 2 Instance Segmentation Function Description

1.Introduction to Instance Segmentation

Instance segmentation is a computer vision technique that simultaneously identifies the position of objects in an image and outlines their contours. In this process, each detected object is assigned a unique mask. A mask can be visualized as a "label layer" with the same dimensions as the original image. These masks not only aid the model in achieving visual comprehension but also enable subsequent tasks like image segmentation and object area measurement.

Under the Instance Segmentation feature of HUSKYLENS 2, the module can recognize up to 80 distinct objects and accurately mark their contours. The recognition categories supported are identical to those of the Object Recognition function.

2.Instance Segmentation Function Description

In this section, we will learn how to use HUSKYLENS 2 to perform instance segmentation on objects present in the image (the object must belong to one of the 80 categories).

2.1 Selecting the Instance Segmentation Function

Power on HUSKYLENS 2, wait until it starts successfully, and find the "Instance Segmentation" function.

2.2 Observe Instance Segmentation Results

To perform instance segmentation on an object: Align the HUSKYLENS 2 with the object (which must belong to one of the 80 categories), ensuring the object is completely within the crosshair area in the frame. Observe the screen; the crosshair will disappear, and a white mask will appear, outlining the object's contour. The object's category label and confidence score will also be displayed.

Learn the object, observe the result: After the category label appears, press Button-A to learn it. Once learning is complete, the white mask changes color, and the learned ID appears.

3.Instance Segmentation Parameter Settings

The factory default parameters of HUSKYLENS 2 meet basic functionality requirements. For more refined features, individual parameters can be adjusted manually.
The following parameters are all based on the "Instance Segmentation" function, so first ensure you have entered the "Instance Segmentation" function, as shown. To modify a parameter, slide left or right on the parameter options at the bottom of the screen to select it.

3.1 Forgetting ID

To forget all previously learned object categories: Step 1, tap "Forget ID" on the screen; Step 2, when the pop-up "Forget all IDs and names" appears, tap "Yes".
At this point, aim the HUSKYLENS 2 at the previously learned yet forgotten object. The left side of the screen will only display the category label, and the mask will turn white, indicating the forgetting process is complete.

3.2 Detection Threshold

The threshold is a confidence bound set during the Instance Segmentation function. Only when the model's prediction confidence exceeds this threshold will the instance segmentation result for that category be output.

Setup Steps: Click "Detection Threshold", and a parameter adjustment slider appears above it. Sliding left reduces the value; sliding right increases it. The effect is shown in the figure.

3.3 Mask Threshold

Mask Threshold controls the mask boundary of recognized objects, allowing adjustment of the mask size range for the recognized objects.
When the mask threshold is small, the mask boundary of the segmented object becomes larger, leading to inaccurate object contours. When the mask threshold is increased, the corresponding object contour boundary becomes more precise.

Setting Steps: Click "Mask Threshold", and a parameter adjustment slider appears above it. Sliding left decreases the value, while sliding right increases it. The effect is shown in the figure below.

3.4 NMS Threshold

NMS Threshold is a common parameter in visual recognition used to filter detection boxes. In visual recognition tasks, the model often predicts multiple overlapping detection boxes around the same target.

Without filtering, a single object may be enclosed by multiple overlapping boxes, which can be resolved by adjusting the NMS threshold to remove overlapping duplicates and retain only the optimal one.

To put it simply, the NMS threshold determines how much overlap between two boxes is considered "duplicate". For example, a low threshold (e.g., 0.3) will mark boxes as duplicates if they overlap slightly, removing one. A high threshold (e.g., 0.7) requires significant overlap to be considered duplicates, resulting in more retained boxes.

Setting Steps: Click "NMS Threshold", and a parameter adjustment slider will appear above it. Sliding left reduces the threshold, resulting in fewer framed objects; sliding right increases the threshold, resulting in more framed objects.

3.5 Set Name

This parameter allows naming recognized barcodes, which can be in both Chinese and English.

Setting Steps: Click "Set Name", slide the number in the top-left corner up/down to select the ID to name; tap the on-screen keyboard to enter the name (as shown in the left figure); after input, click the checkmark in the bottom-right corner to save. A green checkmark will appear in the top-right corner upon successful saving. The following image shows the effect after setting a name.

3.6 Display Name

This parameter controls whether the name of a learned barcode is displayed when recognized, with the default set to display.

Setting Steps: Click "Display Name". If the switch above is blue, it is in the ON state; when a learned barcode is recognized, its name will be displayed (as shown in the left image). Click the switch to turn it white, which is the OFF state; when a learned barcode is recognized, its name will not be displayed (as shown in the right image).

3.7 Restore Defaults

This parameter can restore all settings to their default state and forget learned tags and IDs, but it does not clear exported models (see below for information on exporting models).

Setting Steps: Click "Restore Defaults". A "Restore Default Configuration" dialog box will appear, then click "Yes".

3.8 Export Model

This parameter allows you to save and export the current settings, learned category labels, and IDs to the local memory of HUSKYLENS 2. It is ideal for scenarios such as migrating parameters to another HUSKYLENS 2. This operation does not require inserting a TF card.

Export Steps: Click "Export Model". When the "Save Configuration To" pop-up appears, slide the number selector up and down to choose which model to save as (up to 5 models can be saved), then click the "Confirm" button in the lower-left corner of the pop-up to save. After confirmation is complete, the export process will automatically start, as shown in the figure.

To view the exported model: After the "Exporting" pop-up disappears, you can view the exported model file on your computer. First, connect the HUSKYLENS 2 to your computer's USB port.

Next, access the memory of HUSKYLENS 2 via the path shown in the figure below on your computer. Locate model files with the .bin extension, where the number before the extension indicates the "model number" selected when saving the configuration. This file can be copied and pasted to other locations.

3.9 Importing Model

This parameter allows HUSKYLENS 2 (referred to as "Husky A") to import its model to another HUSKYLENS 2 (referred to as "Husky B"), enabling Husky B to recognize the ID of the barcode learned by Husky A without re-learning.

Importation Steps:

Step 1: Connect Husky A to the computer, then copy the exported model file to the computer's desktop.

Step 2: Connect Husky B to the computer, and paste the file from the previous step into Husky B's specified folder (as shown in the figure). (If the "object-segmentation" folder is not found, first perform

Step 3: after importing the model, the folder will be automatically created. Then return to Step 2.)

Step 4: First, confirm you are in the "Object Classification" function. Then click "Import Model". Once the "Load Configuration" pop-up appears, slide the number up and down to select the model to load.

This should match the model number saved in the previous step. For example, if the model file pasted into HUSKYLENS 2 is repo1.bin, select number 0. Finally, click "Yes" to import.

Then you can perform instance segmentation tests. The left image below shows the instance segmentation result before Husky B imports the model, and the right image shows it after.