3D TOF Sensor Camera

Introduction

The 3D TOF depth camera is composed of BL702 and Juyou 100x100 TOF, and is a 3D sensor that can measure distances up to 1.5m. Its built-in 1.14-inch LCD screen allows for real-time preview of depth maps with colormap. The module is plug-and-play with UART and Type-C interfaces. It supports ROS systems (ROS1+ROS2), enabling fast acquisition of depth data and depth maps. An integrated, cost-effective, and user-friendly 3D Time-of-Flight (ToF) solution designed specifically for Internet of Things (IoT) applications. This solution encompasses various algorithms that enable distance measurement, multi-zone body positioning sensing, posture detection, posture control, and essential calibration functionalities. These features find wide-ranging applications in numerous fields, including white goods, laser televisions, smart projectors, intelligent lighting, smart parking systems, and home automation facilities.

Tutorial

Preparation

Uses the serial port protocol to provide interfaces and transmit data.

The physical interfaces we provide include type-c (virtual serial port) and 1.0mm 4pin motherboard (UART), which can obtain depth map data for integration.

Product wiring instructions

After connecting to the computer through the type-c interface, it can identify /dev/ttyUSBx (Linux) or COMx (Windows).

The usb2ttl module can also be used to connect the 4-PIN bus according to the hardware pin diagram (device screen face down).

SEN0581	TX	RX	GND	5V
USB-TTL	RX	TX	GND	5V

Power-up preview

Button function: "Next" on the left and "Switch" on the right.

Screen display: the + word in the center of the screen is the ranging point, and the top right will display real-time xx cm on behalf of the distance ranging. If there is a large area of objects near the lens, "block!" will be displayed on the right of the screen. In other cases open!.

After the device is connected to the power supply, you can preview the depth pseudo-color map in real time on the built-in LCD screen of the device.

PC side preview interaction

Install COMTOOL on the upper computer before previewing and modifying the Settings on the PC.

Windows system:comtool-windows

Linux system:https://github.com/sipeed/MetaSense-ComTool

Note: Windows 7 and below systems need to install the driver, you can go to the FTDI official website to download.

Open the COMTOOL software and select the Graph interface. If there is no Graph above the software, you can add a Graph at the + sign in the upper right corner. If the middle part is blank after creation, double-click MaixSenseLite in the lower left corner

Select a small Port and baud rate in the left port (choose any high baud rate), click open and check USB to receive a large amount of data. Connected will be displayed in the lower left corner

Set the Header to \x00\xFF to correctly parse the image data and observe the depth map to intuitively feel the depth on the 2D flat image.

Comtool configuration instructions

Comtool configuration control description of the host computer

Header:et the identification header.
RawCMD:User can send commands manually (USB and UART serial ports behave the same).
ISP:start stop.
LCD:display the lcd screen on and off.
USB:serial port transmission depth map on and off.
UART:serial port transmission depth map on and off.
ANTIMMI:Automatic anti-multi-machine interference is turned on and off (susceptible to interference, the effect of turning off is better)
Binn:pull down to set BINNING, Baud to set UART baud rate.
X, Y:coordinates:When the checkbox to the left of D is on, the (x,y) distance from the camera is displayed.
Unit:sets the quantization unit:16-bit quantization to 8-bit, the scale is reduced, if the setting is too small, only very close images can be seen clearly.
FPS:Set the output frame rate (it should not be too high, set it reasonably according to the performance of the docking device, reducing the frame rate can reduce the amount of transmitted data)
Ev:Exposure gap control (leftmost represents AE, others are fixed exposure time)

Case

Measure Objects Distance

The distance between the objects is placed to form the difference in depth value. After the module captures the difference, it displays a warm and cold color, a warm color when the distance is close, and a cool color when the distance is far.

Gesture Interaction

Realize super cool keyboard light follow,track the position of the hand in real time,and then map the keyboard light according to the position of the hand.

Related code

Secondary development Serial protocol

Secondary development manual

Secondary development Access ROS

1.Access ROS1

Preparations

First, prepare the applicable environment: Linux system.

You can use the virtual machine virtual box or vmware.

Install and run

ROS Access Package: sipeed_tof_ms_a010_ros_v0.3.zip

# Unzip sipeed_tof_ms_a010.zip and enter the directory
cd ros1
source /opt/ros/*/setup.sh
catkin_make
source devel/setup.sh 
rosrun sipeed_tof_ms_a010 a010_publisher _device:="/dev/ttyUSB0"
# After that, the terminal will continue to refresh and display [sipeed_tof]: Publishing, that is, it works normally

You can view the frame rate in RQT

RVIZ2 Preview

After opening rviz2, in the lower left corner of the interface Add->By topic->PointCloud2 or /depth ->Image Add ->Display/Global Options/Fixed Frame needs to be modified to tof, the point cloud can be displayed normally. According to the added content, Image will be displayed on the left and the point cloud will be displayed in the middle.

2.Access ROS2