Release notes

We are pleased to announce the availability of version 0.10.0 of Live Sense SDK for Linux.

This is a stable release. We are constantly working on new features, stabilizing APIs, improving our KPIs, and evaluating your feedback.

Changes

This section documents major changes to each release that may require you to update your application.

Version 0.10.1

Improvements

• Improved the detection of stop signs by Road Signs model.

Known issues

This topic describes ongoing issues or bugs and the suggested workarounds:

• Using a lower confidence threshold will result in more detections, but an overall lower accuracy. This can sometimes cause the labels to show false positives. For this reason, applications should keep their confidence threshold at or above 0.6 to maintain reasonably accurate output.
• The accuracy of TensorFlow Lite quant models is lower than that of fp16 or fp32 models.
• The first run for TensorRT involves building the cache for invoking the models. This process takes 40-50 minutes on average to complete on Jetson Nano and about 15-20 mins on Jetson Xavier.
• During the first run for TensorRT, the cache is built, which is required to run the models. During this process, the app may crash. In this case, rerun the app till the cache is built successfully, and then the app will continue to run smoothly.
• The distance and relative position calculation for detections is in Beta, the returned distance and relative position values may be inaccurate.
• Lower accuracy for detections is noted for TensorFlow Lite quantized and EdgeTPU variants of pothole and cone_barrier models.
• The Observation library has limited functionality due to the HERE Data Hub shutdown.

Changes from previous releases

This section documents major changes from past releases. We encourage you to use the latest release, 0.10.1.

Version 0.10.0

New features

• New Road Signs model
  • Additional sign types:
    • Advisory speed limits
    • No access
    • Pedestrian crossing
    • Parking restriction
    • Stop here on red
    • No left turn
    • No U turn
    • No U or left turn
    • No right or left turn
  • Not available in the TensorRT, and EdgeTPU variants of the SDK.
• Added support for dynamic region of interest derived from lane markings.
• Added lane departure alerts.

Improvements

• Improved the distance and relative position calculation for detections (Beta).
• Updated speed bumps quant model.
  • Not available in the TensorRT variant of the SDK.

API changes

• Added setRoi to ls::LSDAlertManager to set a custom RoI for alerts
• Added ls::Roi and ls::Roi::RoiType
• Added determineLaneAlerts to ls::LSDAlertManager
• Added LANE_DEPARTURE to ls::LSDAlertType
• Added bool useLaneDeparture to ls::LSDAlertSettings (default false)
• Added ls::isTrapezoid utility method

Functional and behavioral changes

• Updated entry alerts from LSDAlertManager with support for the following detection classes
  • "car", "rider", "bus", "truck", "bicycle", "motorcycle"
• Updated target TensorFlow Lite version to 2.6.2.

Version 0.9.0

New features

• New detection capabilities available for beta testing:
  • Speed bumps - detects white stripes (cross-walk).
    • Not available in the TensorRT, TensorFlow Lite quantized and EdgeTPU variants of the SDK.
  • Height Restriction Signs - detects road signs that have height specified in feet and inches.
    • Not available in the TensorRT and EdgeTPU variants of the SDK.
  • Speed Bumps Signs - detects speed bump signs.
    • Not available in the TensorRT variant of the SDK.
  • Bridges and Tunnels - detects bridges and tunnels.
    • Not available in the TensorRT variant of the SDK.
  • Road Basics night model - detects features in low-light conditions.
    • Supported features - car, motorcycle, person, bus, truck.
    • Not available in the TensorRT variant of the SDK.
• Added relative velocity estimation for Road Basics classes, except traffic light, through the ls::RelativeVelocityEstimator class.
• Added ls::HeuristicDetectionManager class to handle filtered detection.

Improvements

• Improved the TTC alerts.

API changes

• Added ls::LSDAlertManager::setCollisionBufferDistance to set the buffer distance in meters to be used during collision calculations.
• Added relativeVelocity to ls::TrackedRecognition to get relative velocity of the detection.
• Added ls::RelativeVelocityEstimator class to calculate the relative velocity of detections based on changes in their distance.
• Added bool runRoadBasicsNight to ls::RoadBasicsModelOptions.
• Added enum ls::RoadHazardsModelType.
• Updated ls::RoadHazardsModelOptions to replace multiple model boolean values with a single field of type ls::RoadHazardsModelType.
• Updated ls::RoadHazardsModel constructor to accept a vector of options.
• Removed ls::RoadHazardsModel::setRoadHazardsModelOptions.
• Added ls::HeuristicDetectionManager class to handle filtered detection.

Functional and behavioral changes

• Added return of the most recent alerts if determineAlerts is called within the update interval.

Version 0.8.0

New features

• Added distance and relative position calculation (Beta) for detections from the Road Hazard models (cones & barriers, potholes).
  • Not available in the TensorRT variant of the SDK.
• Added distance and relative position calculation (Beta) for detections from the brake light model.
  • Not available in the TensorRT variant of the SDK.
• Added distance and relative position calculation (Beta) for detections from the Road Signs model.
  • Not available in the TensorRT variant of the SDK.

Improvements

• Improved the Road Basics model for TensorFlow Lite and EdgeTPU variants.
• Improved the Road Lanes model for TensorFlow Lite and TensorRT variants.

API changes

• Updated ls::ModelOptions::getNumThreads to a const method

Resolved issues

• Resolved an issue with the distance and relative position calculation for detections from the Road Basics model when run on GPU with the TensorFlow Lite variant.

Version 0.7.0

New features

• Added distance and relative position calculation (Beta) for detections from the Road Basics model.
  • Not available in the TensorRT variant of the SDK.
• Added pedestrian TTC detection alert.

Improvements

• Improved the Road Signs model for TensorFlow Lite and EdgeTPU variants.

API changes

• Added the getInputWidth(), getInputHeight() and getInputDepth() functions to the LivesenseModel interface
• Removed the input size constants from LivesenseModel
• Added relativePosition to Recognition and TrackedRecognition
• Added RelativePositionData in LSOObservation for setting the relative position of the detection from the camera position
• Added SensorInfoManager and SensorInfo for providing sensor information of yaw and pitch for a device
• Added bearing, bearingAccuracy, and locationAccuracy in the LocationCoordinatees struct.
• Added SensorInfo in LSOObservation
• The model-specific option parameters are now passed by reference rather than pointer for both the constructor and the setter.
  • RoadBasicsModel
  • RoadSignsModel
  • RoadHazardsModel
  • BrakeLightModel

    // Updated Road Basics signatures
    RoadBasicsModel(const ls::ModelOptions& options,
                    const RoadBasicsModelOptions& rbModelOptions);

    void setRoadBasicsModelOptions(const RoadBasicsModelOptions& rbModelOptions);

    // Updated Road Signs signatures
    RoadSignsModel(const ls::ModelOptions& options,
                   const RoadSignsModelOptions& rsModelOptions);

    void setRoadSignsModelOptions(const RoadSignsModelOptions& rsModelOptions);

    // Updated Road Hazards signatures
    RoadHazardsModel(const ls::ModelOptions& options,
                     const RoadHazardsModelOptions& rbModelOptions);

    void setRoadHazardsModelOptions(const RoadHazardsModelOptions& rhModelOptions);

    // Updated Brake Light signatures
    BrakeLightModel(const ls::ModelOptions& options,
                    const BrakeLightModelOptions& brakeModelOptions);

    void setBrakeLightModelOptions(const BrakeLightModelOptions& brakeModelOptions);

Version 0.6.0

New features

• Improved Road Signs model
• New Road Lanes model
  • Available for TensorFlow Lite and TensorRT variants only

API changes

• Added sessionId to ls::LSOObservation
• Added ls::LanePoint
• Added ls::LaneDetection
• Added ls::RoadLanesModel
• Added ls::RoadLaneBoundaryTracker
• Added ls::LaneDetection pointer to ls::Recognition
• Added ls::LaneDetection pointer to ls::TrackedRecognition
• Added operator== to ls::BoundingBox
• Removed ls::LanesModel
• Removed ls::LanesSegmentation
• Removed ls::LaneBoundaryTracker

Version 0.5.0

New features

• Added ls::LSDAlertManager

API changes

• Updated StaticDistanceEstimator to accept a vector of std::shared_ptr<TrackedRecognition>

Version 0.4.0

New features

• Improved Road Signs model
• Added ls::StaticDistanceEstimator

API changes

• Changed behavior of ls::Recognition::getTitle() to return the classifier label, if non-empty, instead of the detection label.
• Removed ls::Recognition::setTitle(std::string)
• Added ls::Recognition::getDetectionLabel() and ls::Recognition::setDetectionLabel(std::string).
• Added ls::Recognition::getCombinedLabel() to return a combination of detection and classifier label for display.
• Moved TrackedRecognition.h to livesense/common/ directory
• Added ls::TrackedRecognition::getTitle() to return the classifier label if non-empty or the detection label if there is no classifier label
• Renamed ls::TrackedRecognition::title to ls::TrackedRecognition::detectionLabel
• Updated ls::SpeedLimitDeviationManager::getDeviationForDetection() to take a std::string as input instead of int
• Added ls::FeatureDeviation
• Updated ls::SpeedLimitDeviation to inherit from ls::FeatureDeviation
• Changed ls::LSOObservation::deviation to be a ls::FeatureDeviation
• Updated labels for multiple models
  • Road Signs speed limit labels are now in GFR format
  • stop -> stop-sign
  • person -> pedestrian
  • green/yellow/red -> traffic-light-[green/yellow/red]
  • rectangular_barrier -> rectangular-barrier
  • strip_stand -> strip-stand
  • cylindrical_barrier -> cylindrical-barrier
  • delineator_barrier -> delineator-barrier
  • traffic-cone -> cone

Version 0.3.0

New features

• Added observation library
• Added connected alerts
• Added speed-limit deviation alerts
• Added experimental GPU support in TensorFlow Lite SDK build
• Added labeling of lane boundaries to lane detection output
• Reduced install size

API changes

• Updated all SDK types to now be located in the ls namespace
• Replace brake-light classifier option of Road Basics with ls::BrakeLightModel
• Added an option for setting the number of TensorFlow Lite CPU threads to ls::ModelOptions
• Added geocoordinate and detectionTimestamp properties to ls::Recognition
• Updated ls::LanesModel to return a ls::LanesSegmentation containing multiple segmentation masks
• Updated ls::LaneBoundaryTracker to accept ls::LanesSegmentation as input to trackResults
• Replace common/common.h and common/utils.h with common/CommonUtil.h
• Renamed or removed macros and methods previously found in common/common.h and common/utils.h

Functional and behavioral changes

• (Premium license only) End-user must accept or decline data collection prior to initializing any ML models
• SDK-generated files are now stored in ~/.config/livesense and ~/.cache/livesense instead of the current directory if $HOME is available

Version 0.2.0

New features

• Introduced support for the Coral Edge TPU and Edge TPU devices
• New detection capabilities
  • Lane detection - available for TensorFlow Lite and Coral Edge TPU variants
• Live Sense SDK for Linux now requires a License key for authentication. This can be obtained by contacting your Account Executive.
• Improvements to object tracker for static objects
• Added MultiBoxObjectTracker to track fast moving objects
• Improved debug and error logging information
• Added external camera support via GStreamer and v4l2

API changes

• Ability to configure the confidence of each model individually
• recognizeImage method changes due to the removal of OpenCV libraries from the SDK

Resolved issues

• Added specific label and confidence fields to the Recognition and TrackedRecognition classes for the following features:
  • Traffic Light Status
  • Brake Light Status
  • Road Signs
• Resolved crash when converting two or more models in TensorRT

Version 0.1.0

New features

• Improved Road Signs global model with stop signs included. This contains maximum speed limit signs for all supported regions.
• Brake light ON/OFF detector model
• Traffic light status detection model
• Road Hazards model - Potholes, and cone and barriers model