Monitor Your Metrics

OLP includes Grafana, an open source data visualization and monitoring tool. Grafana allows you to manage the metrics generated by your catalogs and pipelines, create dashboards and alerts.

To access the Grafana dashboards, click on Monitoring & alerts under the Launcher menu.

OLP Launcher menu to open monitoring
Figure 1. OLP Launcher menu to open monitoring

Once in Grafana, click on the Home button on top left to view the metric dashboards.

Metrics and Labels

Metrics and Labels are the main concept of the data model used to store all the time series data.

Metric data is stored in Prometheus as a time series. A time series is a stream of timestamped values belonging to the same metric and the same set of labeled dimensions.

Note: Creating Grafana dashboards

You must request specific metrics from Prometheus using the PromQL macro language when creating a custom dashboard in Grafana.

The time series stored in Prometheus is uniquely identified by its metric name and optional key-value pairs called labels. The metric name specifies the measured characteristic of a system, for instance, on the Platform you have a metric named layer_total that represents the number of existing layers across all catalogs within your organization.

Labels instead add more dimensions to the data, for instance layer_total can be combined with layer_type=versioned label to filter only versioned layers. Another practical application of this concept is the filtering of multiple pipelines reporting on the same metric: all the pipelines are reporting the pipeline_jobs_failed metric and you can filter only the ones of interest using the pipelineId label with the value of the particular pipeline IDs you are interested into.

All these queries are based on the The PromQL (Prometheus Query Language) that lets the user select and aggregate time series data in real time.

Given a metric name and a set of labels, time series are frequently identified using this notation: <metric name>{<label name>=<label value>, ...}.

Labels are arbitrary and available on the fly when a new service or component sends them along. To see a list of labels and their values associated with a metric, for a given time period, you can use the Explorer function in Grafana.

To access the Explorer function, click on the compass icon on the left toolbar of Grafana.

Access Explorer in Grafana
Figure 2. Access Explorer in Grafana

In the dropdown menu select Metrics and type the name of your metric, e.g.: layer_total{}. If you put your cursor between the curly brackets you will get the list of available labels for that metric, in the selected time period.

List of labels for a particular metric
Figure 3. List of labels for a particular metric

Running the he query will return the table with all the labels and their values, for the selected time period.

Labels and their values for a particular metric
Figure 4. Labels and their values for a particular metric

Query Variables

In addition to the method above and to avoid hard-code values on your metric queries, you can use variables in their place. These variables are called Query variable and they allow you to query Prometheus for a list of metrics, labels or label values.

The following functions can be used in the Query input field:

label_names() Returns a list of label names.
label_values(label) Returns a list of label values for the label in every metric.
label_values(metric, label) Returns a list of label values for the label in the specified metric.
metrics(metric) Returns a list of metrics matching the specified metric regex.
List of possible `endpoint_name` values for `incoming_requests_total` metric using Query variable functions
Figure 5. List of possible `endpoint_name` values for `incoming_requests_total` metric using Query variable functions

Official documentation about Templating and Query Variable

Create a simple Dashboard

Create a Dashboard.

Create a Dashboard
Figure 6. Create a Dashboard

Add a Query to the Panel.

Add a Panel
Figure 7. Add a Panel

Type your query, e.g.: the amount of versioned layer within the realm, in the selected time frame.

Type your query
Figure 8. Type your query

As a exercise, replace the query with the following ones to see how the dashboard changes:

  • sum(catalog_total{}) - Amount of catalog within the realm, in the selected time frame.
  • sum(layer_used_bytes{layer_type="versioned"}) - Sum of data volume across all versioned layers (all catalogs).

Time Ranges and Intervals

You can select the dashboard time range by using the dropdown menu in the upper-right corner of a Grafana dashboard. This controls the range of time over which your dashboard widgets and graphs query for their data.

Grafana menu to select a time range
Figure 9. Grafana menu to select a time range

When a new time range is selected, Grafana automatically calculates the optimal display time interval (such as 1 day or 8 hours) for you. This is because when there is more data than can be shown, Grafana can make a more efficient display by grouping data points in larger intervals.

This auto-calculated interval may not be desired for all dashboards. For dashboards that are labeled "w/ Custom Time Interval", you can use the Custom Time Interval dropdown menu in the upper-left corner to select your own.

Grafana menu to select a time interval
Figure 10. Grafana menu to select a time interval

results matching ""

    No results matching ""