Models

The Live Sense models provide an interface for interacting with the underlying machine learning (ML) models used for object detection.

Note

Object Detection/Recognition describes the combination of object classification, identifying objects of interest that exist in an image, and localization by identifying where the object is located in the image. An example is a Live Sense model that identifies a vehicle in the image and provides a bounding box of where the vehicle was identified.

The core function of every model is to take in an RGB image and return a list of detections found within the image.

Each Live Sense model has a set of object classes that it is able to detect. It is up to the developer to determine which objects, and therefore models, are relevant to their use case.

Model Types and Detection Classes

The following table describes the models and their available classes:

Category Model Name Label Recommended Confidence
Road Basics RB_day pedestrian 47
rider 47
bicycle 45
car 46
motorcycle 47
bus 50
truck 45
traffic-light 47
RB_night pedestrian 45
car 40
motorcycle 40
truck 40
Road Alerts BrakeLight car-brake-light-on 80
Road Hazards cone_barrier cone 60
strip-stand 60
rectangular-barrier 75
cylindrical-barrier 60
delineator-barrier 60
pothole pothole 80
speed_bump_signage speed-bump-sign 55
Road Signs RoadSigns DSSpeedLimit[10-130][White_Circle_00/White_Rectangle_01] 60
stop-sign 60

Note

RB_night detects supported features in low light conditions.

Note::

For DS_SpeedLimit, the label suffix [White_Circle_00/White_Rectangle_01] denotes the sign shape.

  • White_Circle_00 denotes a circular sign
  • White_Rectangle_01 denote a rectangular sign.

Models Available in Beta Mode

The models below are available for beta testing:

Category Model Name Label Recommended Confidence
Road Alerts TrafficLightCVModel traffic-light-green 75
traffic-light-red 75
traffic-light-yellow 75
Road Hazards bridge_tunnel bridge 75
tunnel 75
signage roadworks-going-on 60
road-closed 55
speed_bump_object cross-walk 55
height_restriction_signs height-restriction-sign-Xft-Yin 60
Lane Detection lane_detector lane N/A
RTG real_time_guidance stop-sign 60
traffic-light 60

Confidence Configuration

Live Sense SDK allows to configure the confidence values of the models as well as the individual classes within each model.

Update the Confidence of a Model

Updates the confidence of all the classes available in the specified model to the provided value.

List<Recognition> recognizeImage(Bitmap bitmap, int sensorOrientation, float minimumConfidence)

For a model, the minimumConfidence here can be used from the Recommended Confidence mentioned in the table described in the Model Types and Detection Classes section.

Update the Confidence of a Class

Updates the confidence of the specified class to the provided value. This supersedes the confidence configured for the parent model.

void addClassMinConfidence(String classLabel, float classMinConfidence)

The classLabel and classMinConfidence can be used from the Label, and respective Recommended Confidence values mentioned in the table described in the Model Types and Detection Classes section.

Object Recognition

A Live Sense Recognition describes the following properties of a detected object:

  • class - What object was detected. For more information, see Model Types and Detection Classes.
  • location - Where the object was found in the image frame.
  • confidence score - A number between 0 and 1 that indicates confidence that the object was correctly detected.

Side crop and Center crop inference

Side crop and Center crop inference involves modification of the source image shape by a model to improve detection output. What this option does depends upon the model, and this option should be disabled if large modifications to the image are made before passing it into the model.

Distance and Position Estimation

Note

This feature is in Beta, the returned distance and relative position values may be inaccurate.

All of the object detection models listed above can provide an estimated distance and position relative to the camera's point of view for each detected object.

The output values for the distance and relativePosition properties of a detected object are as follows:

  • Height: Distance of the detection from the ground.
  • Lateral: Distance of the detection relative to the center of the camera's view. This value may be positive or negative with a negative value signifying that the detection is towards the left side of the camera's view and vise-versa.
  • Depth: Distance that the device needs to travel forward in a straight light so that the actual distance is equal to the lateral distance.
  • Distance: Actual distance of the detection from the camera's point of view.

All distance values are provided in meters and are used in the following features of Live Sense SDK

  • Time-to-collision alerts
  • Relative velocity estimation

results matching ""

    No results matching ""