Current Usage Metrics

Current Usage Metrics is a default dashboard available to you in Grafana. The following metrics are available for you to monitor your current usage of the HERE platform. The metric in the single-stat panel to the right of the dashboard shows the current value within the last one hour, while the graph panel to the left shows the time series presentation of this value.

Storage Usage Metrics

METRIC DETAILS DESCRIPTION
Versioned layer_used_bytes with layer_type="versioned" and type="data"
storage_used_bytes with storage_type="stream_durable_volume" 		Storage of versioned controlled, Geo-Indexed data
Volatile layer_allocated_bytes with layer_type="volatile" and type="data"
storage_allocated_bytes 		Storage in volatile memory for more performance with small writes and reads
Stream in_throughput_kbps|out_throughput_kbps with layer_type="stream" Capacity allocated to queue Stream data
Stream TTL ttl_kb_allocated with layer_type="stream" Capacity allocated to store long lived streaming jobs. This covers the EBS volumes associated with the Stream Nodes
MetaData catalog_used_metadata_bytes with type="metadata" Storage of meta-data about the data you store

Transfer Usage Metrics

METRIC DETAILS DESCRIPTION
Log Search IO splunk_index_usage_bytes_previous_hour with operation="splunk_index" Storage of indexed log data for debugging search
Pipeline IO This metric is computed using:
internet_ingress_bytes_total
internet_egress_bytes_total 		Network I/O traffic generated by Pipelines
Data Service IO This metric is computed using:
flink_taskmanager_job_task_operator_rlm_dsio_requestBytes_<organization-id>*
flink_taskmanager_job_task_operator_rlm_dsio_responseBytes_<organization-id>*		 Data transfer (I/O) from within and outside of the platform for accessing and storing data.

* organization-id in the following format, if you organization id is olp-here this value will be olp_here.

Compute Usage Metrics

METRIC DETAILS DESCRIPTION
Compute Core pipeline_cpu_core_seconds CPU Core Hours consumed by Batch or Stream Pipelines
Compute RAM pipeline_ram_gb_seconds RAM Hours consumed by Batch or Stream Pipelines

Common Metrics and Labels used for Current Usage

METRIC NAME LABELS
layer_used_bytes billingTag to filter out for a particular billing tag.
catalog to filter out for a particular catalog id.
catalogHrn to filter out for a particular catalog HRN.
clientAppId to filter out for a particular client app ID.
layer to filter out for a particular layer ID.
layer_type to filter out for a particular layer type, possible values index, stream, versioned, volatile, objectstore.
project to filter out for a particular project ID.
type with value set to data.
storage_used_bytes billingTag to filter out for a particular billing tag.
catalog to filter out for a particular catalog id.
catalogHrn to filter out for a particular catalog HRN.
clientAppId to filter out for a particular client app ID.
layer to filter out for a particular layer ID.
storage_type to filter out for a particular storage type, possible value storage_durable_volume.
project to filter out for a particular project ID.
layer_allocated_bytes billingTag to filter out for a particular billing tag.
catalog to filter out for a particular catalog id.
catalogHrn to filter out for a particular catalog HRN.
clientAppId to filter out for a particular client app ID.
layer to filter out for a particular layer ID.
layer_type to filter out for a particular layer type, in this metric volatile is the only value.
project to filter out for a particular project ID.
ttl_kb_allocated billingTag to filter out for a particular billing tag.
catalog to filter out for a particular catalog id.
catalogHrn to filter out for a particular catalog HRN.
clientAppId to filter out for a particular client app ID.
layer to filter out for a particular layer ID.
layer_type to filter out for a particular layer type, in this metric stream is the only value.
project to filter out for a particular project ID.
catalog_used_metadata_bytes billingTag to filter out for a particular billing tag.
catalog to filter out for a particular catalog id.
catalogHrn to filter out for a particular catalog HRN.
clientAppId to filter out for a particular client app ID.
layer to filter out for a particular layer ID.
project to filter out for a particular project ID.
type with value set to metadata.
splunk_index_usage_bytes_previous_hour billingTag to filter out for a particular billing tag.
clientAppId to filter out for a particular client app ID.
operation with value set to splunk_index.
project to filter out for a particular project ID.
internet_ingress_bytes_total
internet_egress_bytes_total 		app to filter out for the framework app used, values can be flink-1-x and spark-1-x.
billingTag to filter out for a particular billing tag.
jobId to filter out for a particular job ID.
pipelineIdto filter out for a particular pipeline ID.
pipelineVersionId to filter out for a particular pipeline version ID.
pipeline_cpu_core_seconds
pipeline_ram_gb_seconds 		billingTag to filter out for a particular billing tag.
clientAppId to filter out for a particular client app ID.
pipelineDeploymentIdto filter out for a particular pipeline deployment ID.
pipelineFrameworkto filter out for a particular pipeline framework. Currently, the possible values are: FLINK_1_7_1, FLINK_1_X, SPARK_2_1_1, SPARK_2_4_0 .
pipelineIdto filter out for a particular pipeline ID.
pipelineTemplateId to filter out for a particular pipeline template ID.
pipelineVersionId to filter out for a particular pipeline version ID.
project to filter out for a particular project ID.

results matching ""

    No results matching ""