catalog layer stream

The OLP CLI supports the following:

• get partitions from a stream layer
• put partitions to a stream layer
• ingest partitions to a stream layer

catalog layer stream get

Subscribes to the specified stream layer and downloads or prints every partition that is uploaded to the layer during the subscription period.

olp catalog layer stream get <catalog HRN> <layer ID> [command options]

Required parameters:

• <catalog HRN> The HRN of the catalog.
• <layer ID> The ID of the layer.

Optional parameters:

• --output <path to dir> The output directory to store data received from the stream. Files are created with the names of partitions received from the stream. If the specified directory doesn't exist, it is automatically created. If no directory is specified, partitions from the stream are printed to the standard output separated by a delimiter.

  Note

  The command can produce empty output when trying to write the content of the partition that contains non-ASCII symbols to stdout. This is a Windows-specific problem, related to the peculiarity of rendering UTF-8 symbols on Windows console. As a workaround, please redirect command output to a file or specify the output directory with the --output parameter.

• --delimiter <character> A delimiter used for printing stream data to the standard output; empty by default.

• --limit <max number of paritions> The maximum number of partitions to be downloaded.
• --timeout <duration in seconds> The subscription timeout duration in seconds (infinite by default).
• --consumer-group <consumer ID> The name of a consumer, or a group if the consumer is distributed.
• --decode <true|false> Determines whether the partition content is decoded from protobuf and converted to JSON on-the-fly. The default value is false, which means the content is not decoded. You can specify the path to a schema on a local file system with --decode <true|false:[path/to/schema.zip]>; otherwise, the default schema linked to the layer is used. An exception is thrown when there is no schema linked to the layer or no path to a local schema ZIP provided. Schemas can be downloaded from the HERE platform portal or with the olp schema get command. Using --decode true with --output parameter produces files where the following name pattern is used: <partition-name>-decoded.json.
• --credentials <path to credentials file> The name of a credentials file to use with the command. Credentials files are downloaded separately from the platform portal.
• --profile <profile name> The name of the credentials profile to use from the olpcli.ini file.
• --quiet Displays the absolute path to the output directory, if available. Otherwise, the partitions' contents are printed from the stream to the standard output.
• --scope <project HRN> Specifies the project HRN to use as the scope in the request. The value specified with --scope overrides any value for here.token.scope provided in the credentials file used for the command.

For more information on using credentials and profiles, see Credentials setup.

Example:

The command below prints all partitions from stream-layer1 in the catalog1 using the default empty delimiter.

olp catalog layer stream get hrn:here-cn:data::org:catalog1 stream-layer1

catalog layer stream put

Publishes the content from a local machine into a catalog's stream layer.

olp catalog layer stream put <catalog HRN> <layer ID> [command options]

Required parameters:

• <catalog HRN> The HRN of the catalog.
• <layer ID> The ID of the layer.

Optional parameters:

• --input <path to dir> The input directory from where to read partitions. Relative paths are supported. Partitions are populated by walking the file tree rooted at the given input directory. The file tree is traversed depth-first and in lexicographical order. If not specified, the partitions are taken from the standard input and split by the delimiter.
• --delimiter <character> A delimiter used for reading partitions from the standard input.
• --limit <max number of partitions> The maximum number of partitions to be published.
• --encode <true|false:[path/to/schema.zip]> Determines whether the partition content is encoded from JSON and converted to protobuf on-the-fly. The default value is false, which means the content is not encoded. Supported layer content types: protobuf. You can specify the path to a schema on a local file system; otherwise, the default schema linked to the layer is used. An exception is thrown when there is no schema linked to the layer or no path to a local schema ZIP provided. Schemas can be downloaded from the HERE platform portal or with the olp schema get command.

• --credentials <path to credentials file> The name of a credentials file to use with the command. Credentials files are downloaded separately from the platform portal.
• --profile <profile name> The name of the credentials profile to use from the olpcli.ini file.
• --quiet Displays empty output with no additional information.
• --scope <project HRN> Specifies the project HRN to use as the scope in the request. The value specified with --scope overrides any value for here.token.scope provided in the credentials file used for the command.

For more information on using credentials and profiles, see Credentials setup.

Example:

The command below publishes all partitions from /path/to/directory.

olp catalog layer stream put hrn:here-cn:data::org:catalog1 stream-layer1 --input /path/to/directory

catalog layer stream ingest

Publishes the content from a local machine into a catalog's stream layer.

olp catalog layer stream ingest <catalog HRN> <layer ID> [command options]

Required parameters:

• <catalog HRN> The HRN of the catalog.
• <layer ID> The ID of the layer.

Optional parameters:

• --input <path to dir> The input directory from where to read partitions. Relative paths are supported. Partitions are populated by walking the file tree rooted at the given input directory. The file tree is traversed depth-first and in lexicographical order. If not specified, the partitions are taken from the standard input and split by the delimiter.
• --delimiter <character> The delimiter used for reading partitions from the standard input.
• --limit <max number of partitions> The maximum number of partitions to be published.
• --billing-tag <tag string> A tag for cost allocation used to group billing records.
• --x-here-trace-id <trace ID> The trace ID used to track this ingestion session. If skipped, a random UUID is generated instead.
• --credentials <path to credentials file> The name of a credentials file to use with the command. Credentials files are downloaded separately from the platform portal.
• --profile <profile name> The name of the credentials profile to use from the olpcli.ini file.
• --quiet Displays empty output with no additional information.
• --scope <project HRN> Specifies the project HRN to use as the scope in the request. The value specified with --scope overrides any value for here.token.scope provided in the credentials file used for the command.

For more information on using credentials and profiles, see Credentials setup.

Example:

The command below publishes all partitions from /path/to/directory.

olp catalog layer stream ingest hrn:here-cn:data::org:catalog1 stream-layer1 --input /path/to/directory