Run the example applications

Example applications are bundled with the Debian package. If the example applications are not available, contact your HERE representative.

Example source code to demonstrate the API, and make predictions over a sample data, is provided in <LIVESENSE-INSTALL-DIR/example/ folder.

After installing the Debian package, you can find the source code for the examples in /usr/local/share/livesense/example.

The following dependency is required before you run these applications:

sudo apt-get install libssl-dev

# OpenCV:
# OpenCV is used within the example applications as a matter of convenience.
# OpenCV is not required by the {{book.standard.productNameSdk}}

# GStreamer:
# gstreamer development libraries are required to compile and link the
# example_camera_viewer example application. For more information,
# see the GStreamer installation documentation for your
# (Debian-based) system.

To explicitly compile the provided sample, do the following:

Step 1: Provide access to the example folder for your user

sudo chown -R $USER:$USER example/

Step 2: Set credentials

For details on obtaining credentials, see Authenticating Applications.

# Setup environment variables or modify example/CMakeLists.txt
export LIVE_SENSE_APP_ID=<app-id>
export LIVE_SENSE_APP_TOKEN=<app-token>
export LIVE_SENSE_LICENSE_KEY=<license-key>

Step 3: Compile the .cpp files via cmake

mkdir build && cd build
cmake ..
# OR if using the TensorRT variant
# cmake -DTENSORRT_LIB=ON ..
make

Step 4: Run the executable

  export DISPLAY=:{Your_display_identifier}  
  export LD_LIBRAY_PATH=/usr/local/lib/

 ./<example-program> [example-args]

Please refer to the README.md provided alongside the example applications for more details.

You can use the following images to test the models:

• Road Basics model

  

• Road Hazards model

  

• Road Signs model

  

• Road Signs model

  

Model and tracker examples

Several example applications are provided to demonstrate the use of models and trackers available within the SDK on an input video. Each of these examples plots the model outputs onto the video frames with the annotated frame both displayed on screen and written to an output video for review.

These examples include the following:

• example_without_tracker - Feature detection by Road Basics, Road Signs, and Road Hazard models
• example_with_tracker - Feature detection and tracking with asynchronous model inference
• example_with_lanes - Lane detection
• example_with_lanes_tracker - Lane detection and tracking with asynchronous lane model inference

Connected capability examples

Support for external camera sensors using GStreamer

Real-time image capture from external cameras is provided by the Live Sense SDK for Linux with the GStreamer Multimedia Framework. As noted above, the GStreamer development libraries and header files are required to run the example_camera_viewer.cpp example application. For more information, see the GStreamer installation documentation appropriate for your (Debian-based) system.

If running the Live Sense SDK for Linux on an NVIDIA Jetson platform, the example program will target the hardware-accelerated GStreamer libraries and device drivers from NVIDIA.

Otherwise, the v4l2 device drivers and API will be used for real-time capture.

Consider installing the v4l2-utils package to query your system for camera devices:

sudo apt-get install v4l-utils
# List Devices and Formats
v4l2-ctl --list-devices --list-formats

# Show the Driver Info
v4l2-ctl -d /dev/video0 -D
v4l2-ctl -d 0 -D

# List Formats and Resolutions for a Device
v4l2-ctl --list-formats-ext --device /dev/video0

Example stdout with a v4l2-support device at /dev/video2

$ ./example_camera_viewer
2020-11-02 11:06:58.649 [INFO ] [gstreamer] - initialized gstreamer, version 1.14.5.0
2020-11-02 11:06:58.649 [INFO ] [gstreamer] - GstCamera: attempting to create device v4l2:///dev/video2
2020-11-02 11:06:59.582 [INFO ] [gstreamer] - GstCamera pipeline string:
2020-11-02 11:06:59.582 [INFO ] [gstreamer] -   v4l2src device=/dev/video2 ! video/x-raw, format=(string)YUY2, width=(int)640, height=(int)360 ! videoconvert ! video/x-raw, format=(string)RGB, width=(int)640, height=(int)360 ! appsink name=mysink
2020-11-02 11:06:59.587 [INFO ] [gstreamer] - GstCamera successfully created device v4l2:///dev/video2
2020-11-02 11:06:59.587 [INFO ] [camera-viewer] - Successfully initialized camera device (640x360)
2020-11-02 11:06:59.587 [INFO ] [gstreamer] - opening GstCamera for streaming, transitioning pipeline to GST_STATE_PLAYING
2020-11-02 11:06:59.587 [INFO ] [gstreamer] - gstreamer changed state from NULL to READY ==> mysink
2020-11-02 11:06:59.587 [INFO ] [gstreamer] - gstreamer changed state from NULL to READY ==> capsfilter1
2020-11-02 11:06:59.587 [INFO ] [gstreamer] - gstreamer changed state from NULL to READY ==> videoconvert0
2020-11-02 11:06:59.587 [INFO ] [gstreamer] - gstreamer changed state from NULL to READY ==> capsfilter0
2020-11-02 11:06:59.587 [INFO ] [gstreamer] - gstreamer changed state from NULL to READY ==> v4l2src0
2020-11-02 11:06:59.587 [INFO ] [gstreamer] - gstreamer changed state from NULL to READY ==> pipeline0
2020-11-02 11:06:59.587 [INFO ] [gstreamer] - gstreamer changed state from READY to PAUSED ==> capsfilter1
2020-11-02 11:06:59.587 [INFO ] [gstreamer] - gstreamer changed state from READY to PAUSED ==> videoconvert0
2020-11-02 11:06:59.587 [INFO ] [gstreamer] - gstreamer changed state from READY to PAUSED ==> capsfilter0
2020-11-02 11:06:59.587 [INFO ] [gstreamer] - gstreamer stream status CREATE ==> src
2020-11-02 11:06:59.587 [INFO ] [gstreamer] - gstreamer changed state from READY to PAUSED ==> v4l2src0
2020-11-02 11:06:59.587 [INFO ] [gstreamer] - gstreamer changed state from READY to PAUSED ==> pipeline0
2020-11-02 11:06:59.587 [INFO ] [gstreamer] - gstreamer changed state from PAUSED to PLAYING ==> capsfilter1
2020-11-02 11:06:59.587 [INFO ] [gstreamer] - gstreamer changed state from PAUSED to PLAYING ==> videoconvert0
2020-11-02 11:06:59.587 [INFO ] [gstreamer] - gstreamer changed state from PAUSED to PLAYING ==> capsfilter0
2020-11-02 11:06:59.587 [INFO ] [gstreamer] - gstreamer changed state from PAUSED to PLAYING ==> v4l2src0
2020-11-02 11:06:59.587 [INFO ] [gstreamer] - gstreamer stream status ENTER ==> src
2020-11-02 11:06:59.687 [INFO ] [camera-viewer] - Camera open for streaming
2020-11-02 11:06:59.834 [WARN ] [gstreamer] - GstCamera: map buffer size was less than max size (691200 vs 691207)
2020-11-02 11:06:59.834 [DEBUG] [gstreamer] - GstCamera: received first frame, codec=raw format=rgb8 width=640 height=360 size=691207
2020-11-02 11:06:59.834 [INFO ] [gstreamer] - gstreamer changed state from READY to PAUSED ==> mysink
2020-11-02 11:06:59.834 [INFO ] [gstreamer] - gstreamer changed state from PAUSED to PLAYING ==> mysink
2020-11-02 11:06:59.834 [INFO ] [gstreamer] - gstreamer changed state from PAUSED to PLAYING ==> pipeline0
^Creceived SIGINT
2020-11-02 11:07:07.416 [INFO ] [camera-viewer] - Shutting down...
2020-11-02 11:07:07.416 [ERROR] [gstreamer] - GstCamera: stopping pipeline, transitioning to GST_STATE_NULL
2020-11-02 11:07:07.668 [INFO ] [gstreamer] - GstCamera: pipeline stopped
2020-11-02 11:07:07.669 [INFO ] [camera-viewer] - Shutdown complete.

Center crop

A helper class ImageCropper is provided with examples for optional cropping operations performed before sending an image to Live Sense models. The cropping can provide more detail of target objects to said models, particularly with smaller and/or distant objects such as signs. The following two cropping operations are available:

• Side cropping to remove parts of the frame from the sides based on the given percentages.

• Progressive center cropping to return a percentage of the image around the center of the image. The crop sequence is 30-percent, then 50-percent, and finally 100-percent. This loop is then repeated for the following frames.

  By default, no cropping is performed. Center cropping is performed after any side crop is applied.

Detection recording examples

The RecordingHelper class provided alongside the example applications contains methods to save an image along with JSON files containing details of any objects detected in the image.

The following examples demonstrate the use of the RecordingHelper class:

• example_recording_helper - Detection recording for Road Basics and Road Signs
• example_recording_helper_with_lanes - Detection recording for lane markings

Distance logger

A helper class DistanceLogger is provided alongside the example applications for optional logging of traveled distances to a json file.