Basic usage

Before proceeding, make sure you have followed the steps described in Add SDK to a Project and Authenticating Applications.

The most basic use of Live Sense SDK for Linux includes the detection of cars, pedestrians, signs, and other supported objects in a still image. For details on what can be detected by each model, see Models.

This feature can be used for running a single model or multiple models. To enable this type of usage, do the following:

// Live Sense namespace
using namespace ls;

// Initialize Live Sense SDK.
LivesenseEngine::init();

// Activate one of the below options to run the model with precision fp16 or int8.
// ENABLE ONLY 1 of fp16 or int8, or leave both false to use fp32
ModelOptions mOptions;
mOptions.setRunInFp16(true);
mOptions.setRunInInt8(true);

// Initialize options for road basics model.
RoadBasicsModelOptions roadBasicsModelOptions;

// Activate traffic light model, default is off.
roadBasicsModelOptions.runTrafficLight = false;

// Activate road basics night model, default is off.
roadBasicsModelOptions.runRoadBasicsNight = false;

//Setting the confidence threshold for individual models.
roadBasicsModelOptions.roadBasicsConfidenceThresh = 0.55;
roadBasicsModelOptions.trafficLightConfidenceThresh = 0.55;

// Initialize road basics model.
RoadBasicsModel roadBasicsModel(mOptions, roadBasicsModelOptions);

// Initialize options for road signs model.
RoadSignsModelOptions roadSignsModelOptions;
//Setting the confidence threshold for RoadSignsModel
roadSignsModelOptions.roadSignsConfidenceThresh = 0.6;
// Initialize road signs model.
RoadSignsModel roadSignsModel(mOptions, roadSignsModelOptions);

// Initialize options for road hazards model.
// Each distinct `RoadHazardsModelType` in the vector will be loaded by `RoadHazardsModel` 
std::vector<ls::RoadHazardsModelOptions> roadHazardsModelOptions = {
    {ls::RoadHazardsModelType::RH_CONE_CONSTRUCTION, 0.6},
    {ls::RoadHazardsModelType::RH_POTHOLES, 0.6},
    {ls::RoadHazardsModelType::RH_SPEED_BUMP_OBJECT, 0.55},
    {ls::RoadHazardsModelType::RH_SPEED_BUMP_SIGNAGE, 0.55},
    {ls::RoadHazardsModelType::RH_BRIDGE_TUNNEL, 0.75},
    {ls::RoadHazardsModelType::RH_HEIGHT_RESTRICTION_SIGNS, 0.6}};
// Initialize road hazards model
RoadHazardsModel roadHazardsModel(mOptions, roadHazardsModelOptions);


// Initialize options for brake light model.
BrakeLightModelOptions brakeLightModelOptions;
// Initialize brake light model.
BrakeLightModel brakeLight(mOptions, brakeLightModelOptions);

// cv::Mat `frame` is the image to process
cv::Mat frameRGB;
cv::cvtColor(frame, frameRGB, COLOR_BGR2RGB);

// The input frame is required to be of colorspace `RGB`, and be resized to
// aspect ratio expected by `recognizeImage()`. This aspect ratio is
// LivesenseModel::getInputWidth() x LivesenseModel::getInputWidth().
// OpenCV is the recommended library for such input image transformations.
// However, Live Sense also provides resize functions via ImageTransform in the
// event that OpenCV is not available.

unsigned char* origInput = &frameRGB.data[0];
int origWidth = frameRGB.cols;
int origHeight = frameRGB.rows;


int resizedInputWidth = roadBasicsModel.getInputWidth();
int resizedInputHeight = roadBasicsModel.getInputHeight();
int numChannels = roadBasicsModel.getInputDepth();
unsigned char* resizedInput = /* Resized input frame */

// Get recognitions for the road basics model. Note: the original frame is
// optional here but required if brake light and traffic signal classification
// is enabled via the above ModelOptions.
auto roadBasicsDet = roadBasicsModel.recognizeImage(resizedInput,
                                                    resizedInputWidth,
                                                    resizedInputHeight,
                                                    origWidth,
                                                    origHeight,
                                                    origInput);

// Each model may have its own input shape. For code to assist in
// managing the different input shapes, see the example applications
// provided alongside the SDK.
resizedInputWidth = roadSignsModel.getInputWidth();
resizedInputHeight = roadSignsModel.getInputHeight();
numChannels = roadSignsModel.getInputDepth();
resizedInput = /* Resized input frame */

// Get recognitions for the road signs model. Note: the original frame is
// required for proper and accurate sign classification.
auto roadSignsDet = roadSignsModel.recognizeImage(resizedInput,
                                                  resizedInputWidth,
                                                  resizedInputHeight,
                                                  origWidth,
                                                  origHeight,
                                                  origInput);

// Get shape for the next model and resize frame
resizedInputWidth = roadHazardsModel.getInputWidth();
resizedInputHeight = roadHazardsModel.getInputHeight();
numChannels = roadHazardsModel.getInputDepth();
resizedInput = /* Resized input frame */

// Get recognitions for the road hazards model. Note: the original
// frame is not passed in as no classifications are executed.
auto roadHazardsDet = roadHazardsModel.recognizeImage(resizedInput,
                                                      resizedInputWidth,
                                                      resizedInputHeight,
                                                      origWidth,
                                                      origHeight);

// Get shape for the next model and resize frame
resizedInputWidth = brakeLight.getInputWidth();
resizedInputHeight = brakeLight.getInputHeight();
numChannels = brakeLight.getInputDepth();
resizedInput = /* Resized input frame */

// Get recognitions for the brake light model. Note again the original
// frame is not passed in as no classifications are executed.
auto brakeLightDet = brakeLight.recognizeImage(resizedInput,
                                               resizedInputWidth,
                                               resizedInputHeight,
                                               origWidth,
                                               origHeight);

For recommendations on using the Live Sense SDK for Linux in your application, see Recommendations