Publish Protobuf schema to the platform

Objectives: Generate, build, and publish Protobuf schema to the platform

Complexity: Beginner

Time to complete: 20 min

Prerequisites: Get your credentials, Verify your credentials

The HERE platform allows you to define schemas that are used for sharing and transferring data between other platform users and pipelines. The HERE platform portal uses schema artifacts to dynamically decode data stored on the platform. The platform uses Protobuf files to define these schemas. A schema is a collection of related Protobuf messages (stored in one or multiple .proto files) that describe the structure of a partition in a catalog layer. It is a high-level construct that consists of a set of artifacts.

This tutorial demonstrates how to generate and build a Protobuf schema and publish it to the platform. It shows how to get information about a published Protobuf schema and how to download a Protobuf schema.

The tutorial covers the following topics:

For this tutorial, you need to have the CLI installed and have both platform and repository credentials set up on your local machine.

To verify that you set up your platform credentials correctly, make sure that the Verify your credentials tutorial returns the expected result.

Generate Protobuf schema

There are two ways to generate a Protobuf schema: using the OLP CLI and using a Maven Archetype. In this tutorial, we will use the OLP CLI to generate a Protobuf schema. The OLP CLI provides the olp schema generate command that generates a new Maven project for a schema.

Note

The olp schema generate require Java JDK for Protobuf schema generation. Installing Java JRE instead of JDK will cause schema generation fail

Let's generate a Protobuf schema project in the current directory with the com.here.platform.tutorial group ID and generate_protobuf_schema artifact ID:

olp schema generate com.here.platform.tutorial generate_protobuf_schema --type proto

Note

The group ID should usually include a team/product hierarchy to the group ID, such as com.some-company.automated-driving.schema. This way, different teams can manage their schemas independently.

When you deploy a schema to the Artifact Service, its group ID is being reserved by your organization, and no other realm can reuse it in this case. So, specifying generic group ID values like com.example, com.here.schema, and com.here.example, may make it impossible to upload the schema to the Artifact Service repository and, thus, using it in catalogs or sharing it with other users. The main recommendation is to follow package naming best practices

The OLP CLI should return the following output:

Schema has been successfully generated in /home/user/generate_protobuf_schema.

Once the Protobuf schema is generated, we can take a look at the structure of the generated project.

Protobuf schema structure

The olp schema generate command generates a project with the following structure:

generate_protobuf_schema/
├── ds
│   ├── pom.xml
│   └── src
│       ├── assembly
│       │   └── proto.xml
│       └── main
│           └── resources
│               ├── renderers
│               │   └── ReadMe.txt
│               └── ResourcesReadMe.txt
├── java
│   └── pom.xml
├── proto
│   ├── pom.xml
│   └── src
│       ├── assembly
│       │   └── proto.xml
│       └── main
│           ├── proto
│           │   └── com
│           │       └── here
│           │           └── platform
│           │               └── tutorial
│           │                   └── generate_protobuf_schema
│           │                       └── v1
│           │                           └── generate_protobuf_schema.proto
│           └── resources
│               └── description.md
├── scala_2.12
│   └── pom.xml
├── pom.xml
└── schema.yml

From the structure above, we can see that the generated project contains the ds, java, proto, scala_2.12 modules, schema.yml and pom.xml files.

The ds module contains a data schema (ds) bundle that is used to dynamically decode partitions. You can place a GeoJSON renderer for the schema here. The HERE platform portal uses a renderer to visualize partition data on top of the base map. The ds/src/main/resources/renderers folder is the place where you can put a GeoJSON renderer that becomes bundled with your schema once published to the HERE platform. The renderer is written in JavaScript and is used for visualizing partition data on top of the base map in the HERE platform portal. For more information, see the Implement a GeoJSON Renderer documentation.

The proto module contains the definitions of the Protobuf messages for the schema. The proto/src/main/proto/com/here/platform/tutorial/generate_protobuf_schema/v1/generate_protobuf_schema.proto file is the main Protobuf file that describes the data format:

syntax = "proto3";

package com.here.platform.tutorial.generate_protobuf_schema.v1;

message MainProtobufMessage {
    int32 lat = 1;
    int32 lon = 2;
}

The default main Protobuf file describes the data format including two required properties lat and lon of the int32 type.

The content of the main Protobuf file must conform to the Protobuf syntax.

The proto/src/main/resources/description.md file is an empty file where you can put detailed schema documentation in the Markdown format.

The pom.xml file in the schema project root contains the main schema details, such as name, description, version. If you want to override the default values in the pom.xml, specify the name, description or version parameters when running the olp schema generate command.

Declarations of other schemas that the given schema is dependent on are also included in pom.xml:


<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
    <groupId>com.here.platform.tutorial</groupId>
    <artifactId>generate_protobuf_schema_v1</artifactId>
    <version>1.0.0</version>
    <packaging>pom</packaging>
    <name>Schema</name>
    <description>Schema project for OLP</description>
    <inceptionYear>2019</inceptionYear>
    <distributionManagement>
        <repository>
            <id>HERE_PLATFORM_ARTIFACT</id>
            <layout>here</layout>
            <!-- This custom url serves to invoke the HERE wagon extension that handles OAUTH
                 and re-writes Maven URIs for the HERE secure repository.-->
            <url>here+artifact-service://artifact-service</url>
        </repository>
    </distributionManagement>
    <parent>
        <groupId>com.here.platform</groupId>
        <artifactId>sdk-standalone-bom_2.12</artifactId>
        <version>2.47.2</version>
    </parent>
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
        <protobuf.version>3.11.4</protobuf.version>
        <protoc.version>3.11.4</protoc.version>
        <here.plugin.version>2.1.20</here.plugin.version>
        <artifact.wagon.version>2.0.13</artifact.wagon.version>
    </properties>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>com.google.protobuf</groupId>
                <artifactId>protobuf-java</artifactId>
                <version>${protobuf.version}</version>
                <scope>compile</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>
    <repositories>
        <repository>
            <id>HERE_PLATFORM_ARTIFACT</id>
            <layout>here</layout>
            <url>here+artifact-service://artifact-service</url>
        </repository>
    </repositories>
    <build>
        <pluginManagement>
            <plugins>
                <plugin>
                    <groupId>org.eclipse.m2e</groupId>
                    <artifactId>lifecycle-mapping</artifactId>
                    <version>1.0.0</version>
                    <configuration>
                        <lifecycleMappingMetadata>
                            <pluginExecutions>
                                <pluginExecution>
                                    <pluginExecutionFilter>
                                        <groupId>com.here.platform.schema.maven_plugins</groupId>
                                        <artifactId>proto-unpack-plugin</artifactId>
                                        <versionRange>[0.0.1,)</versionRange>
                                        <goals>
                                            <goal>unpack</goal>
                                        </goals>
                                    </pluginExecutionFilter>
                                    <action>
                                        <execute>
                                            <runOnIncremental>false</runOnIncremental>
                                        </execute>
                                    </action>
                                </pluginExecution>
                                <pluginExecution>
                                    <pluginExecutionFilter>
                                        <groupId>net.alchim31.maven</groupId>
                                        <artifactId>scala-maven-plugin</artifactId>
                                        <versionRange>[0.0.1,)</versionRange>
                                        <goals>
                                            <goal>compile</goal>
                                        </goals>
                                    </pluginExecutionFilter>
                                    <action>
                                        <execute>
                                            <runOnIncremental>false</runOnIncremental>
                                        </execute>
                                    </action>
                                </pluginExecution>
                            </pluginExecutions>
                        </lifecycleMappingMetadata>
                    </configuration>
                </plugin>
                <plugin>
                    <groupId>com.here.platform.schema.maven_plugins</groupId>
                    <artifactId>proto-unpack-plugin</artifactId>
                    <version>${here.plugin.version}</version>
                    <executions>
                        <execution>
                            <id>unpack</id>
                            <phase>validate</phase>
                            <goals>
                                <goal>unpack</goal>
                            </goals>
                        </execution>
                    </executions>
                    <configuration>
                        <outputDirectory>${project.build.directory}/proto</outputDirectory>
                    </configuration>
                </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-source-plugin</artifactId>
                    <version>3.0.1</version>
                    <executions>
                        <execution>
                            <id>attach-sources</id>
                            <phase>verify</phase>
                            <goals>
                                <goal>jar-no-fork</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </pluginManagement>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-antrun-plugin</artifactId>
                <version>1.7</version>
                <inherited>false</inherited>
                <executions>
                    <execution>
                        <id>copy-schema</id>
                        <phase>package</phase>
                        <goals>
                            <goal>run</goal>
                        </goals>
                        <configuration>
                            <target>
                                <echoproperties destfile="${project.build.directory}/build.properties"/>
                                <copy file="${project.basedir}/schema.yml" overwrite="true" todir="${project.build.directory}">
                                    <filterset begintoken="${" endtoken="}" filtersfile="${project.build.directory}/build.properties"/>
                                </copy>
                            </target>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-enforcer-plugin</artifactId>
                <version>3.0.0-M2</version>
                <executions>
                    <execution>
                        <id>enforce-property</id>
                        <goals>
                            <goal>enforce</goal>
                        </goals>
                        <configuration>
                            <rules>
                                <requireProperty>
                                    <property>project.groupId</property>
                                    <message>Project groupId must be specified.</message>
                                    <regex>^[a-zA-Z_]+[a-zA-Z0-9_-]*(\.[a-zA-Z0-9_]+[a-zA-Z0-9_-]*)+$</regex>
                                    <regexMessage>Project groupId must be specified as reverse domain name.</regexMessage>
                                </requireProperty>
                            </rules>
                            <fail>true</fail>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>build-helper-maven-plugin</artifactId>
                <version>1.6</version>
                <inherited>false</inherited>
                <executions>
                    <execution>
                        <id>attach-artifacts</id>
                        <phase>package</phase>
                        <goals>
                            <goal>attach-artifact</goal>
                        </goals>
                        <configuration>
                            <artifacts>
                                <artifact>
                                    <file>${project.build.directory}/schema.yml</file>
                                    <type>yml</type>
                                    <classifier>here-schema</classifier>
                                </artifact>
                            </artifacts>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>flatten-maven-plugin</artifactId>
                <version>1.1.0</version>
                <configuration>
                    <flattenMode>bom</flattenMode>
                    <outputDirectory>${project.build.directory}</outputDirectory>
                    <flattenedPomFilename>${project.artifactId}-${project.version}.pom</flattenedPomFilename>
                </configuration>
                <executions>
                    <execution>
                        <id>flatten</id>
                        <phase>process-resources</phase>
                        <goals>
                            <goal>flatten</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>flatten.clean</id>
                        <phase>clean</phase>
                        <goals>
                            <goal>clean</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
        <extensions>
            <extension>
                <groupId>com.here.platform.artifact</groupId>
                <artifactId>artifact-wagon</artifactId>
                <version>${artifact.wagon.version}</version>
            </extension>
        </extensions>
    </build>
    <modules>  
        <module>proto</module>
        <module>java</module>
        <module>scala_2.12</module>
        <module>ds</module>
    </modules>
</project>

The schema.yml file in the root of the project is a descriptor file that indicates that your project is a schema project:

name: "${project.name}"
summary: "${project.description}"
layerSchema: true
type: "proto"

The name and summary properties define the name and description of your schema, respectively. By default, these values are taken from the pom.xml file. The general rule of thumb is not to change these properties in the schema.yml file and define them only in pom.xml. We recommend following best practices for schema naming and schema description

The layerSchema property defines whether this schema should be used as a standalone schema. Setting it to false means that the schema will only be used as a dependency for another schema and will not be visible on the platform nor listed in the OLP CLI.

Note

The schema won't be visible on the platform if the project was built and deployed without the schema.yml file in the root folder.

Extend with existing Protobuf schema

Once we have generated a Protobuf schema project, we can extend it with any other Protobuf schema.

In this tutorial, we will update the proto/src/main/proto/com/here/platform/tutorial/generate_protobuf_schema/v1/generate_protobuf_schema.proto main Protobuf file by providing an additional property and describing this schema in the documentation file.

Let's start with adding public schema com.here.schema.rib:address-attributes_v2 as a dependency. We don't have to specify the version property, it will be taken from the BOM file of Java/Scala Data SDK. Here you can find a list of available dependencies

Add a dependency to the generate_protobuf_schema/proto/pom.xml file:

<dependency>
  <groupId>com.here.schema.rib</groupId>
  <artifactId>address-attributes_v2_proto</artifactId>
  <type>zip</type>
</dependency>

Add dependency to the generate_protobuf_schema/java/pom.xml file:

<dependency>
  <groupId>com.here.schema.rib</groupId>
  <artifactId>address-attributes_v2_java</artifactId>
  <type>jar</type>
</dependency>

Add dependency to the generate_protobuf_schema/scala_2.12/pom.xml file:

<dependency>
  <groupId>com.here.schema.rib</groupId>
  <artifactId>address-attributes_v2_scala_2.12</artifactId>
  <type>jar</type>
</dependency>

The com.here.schema.rib:address-attributes_v2 schema provides the Address Attribution Model containing addressing information. These models might be used for defining the main schema message. In particular, the schema provides a model that describes postal code com.here.schema.rib.v2.PostalCode defined in file com/here/schema/rib/v2/address_attributes.proto.

Let's add new property postal_code with type com.here.schema.rib.v2.PostalCode to the main Protobuf file. The updated main Protobuf file looks as follows:

syntax = "proto3";

package com.here.platform.tutorial.generate_protobuf_schema.v1;

import "com/here/schema/rib/v2/address_attributes.proto";

message MainProtobufMessage {
  int32 lat = 1;
  int32 lon = 2;
  com.here.schema.rib.v2.PostalCode postal_code = 3;
}

We recommend following best practices for protobuf code styling

The next step is to add documentation about our schema.

You can add prepared documentation when generating a new schema project using the olp schema generate OLP CLI command with the --documentation parameter.

There are two ways to pass your documentation to the schema project:

  • Pass the URL to the documentation if you want to refer to external documentation or specific HERE documentation.
  • Specify the path to a Markdown file if you want to include your own schema description.

If you already have a generated schema project, and you want to update or add documentation, you need to update the proto/src/main/resources/description.md file, or to add/update the docUrl field in the schema.yml file.

Let's add the following documentation to the proto/src/main/resources/description.md file:

This schema was generated for the `Publish Protobuf schema to the Platform` tutorial.


The main protobuf file describes the data format including the following properties:


`lat` - the measurement of distance north or south of the Equator.

`lon` - the measurement east or west of the prime meridian.

`postal_code` - represents the code generated by the government to facilitate mail delivery.

Once our schema project is updated, we need to validate and package the project to make sure that our changes have not broken the project itself.

Validate and package Protobuf schema

The OLP CLI provides the olp schema package command to build and validate the schema project.

The olp schema package command can be useful when you have updated the schema project and want to make sure that your project is valid before deploying it to the platform. You don't need to use this command if you do not change the schema.

During the schema building, the following validation plugins verify that the Protobuf schema project contains a valid main Protobuf file:

  1. The GoogleStyle validator verifies that schemas messages follow the Google's Protobuf Style Guide.
  2. The FileExtension validator verifies that all Protobuf files use the .proto extension.
  3. The MajorVersionInPackage validator verifies that the major version of the artifact is in the package. For example, version 2.3 should have "v2" as part of its package: "com.here.platform.schema.foo.v2".
  4. The BackwardsCompatibility validator verifies that any changes in your .proto files within a given major version are backwards compatible. Breaking changes will require a major version bump. The validator looks for the following changes:
    • File removed
    • Message removed
    • Field removed
    • Field type changed
    • Field label changed
    • Field number changed
    • Enum removed
    • Enum value removed
    • Enum value changed
    • Proto syntax changed
  5. The PackageConsistency validator verifies that the directory tree of the Protobuf source files corresponds to the Protobuf package declaration in the individual files.

You can find the validation results in the proto/target/proto/META-INF/validation-results.json file after the olp schema package command is executed.

Let's build an updated Protobuf schema project using the following OLP CLI command:

olp schema package --input /home/user/generate_protobuf_schema

The OLP CLI should return the following output:

Schema has been packaged successfully.

The message above indicates that your Protobuf schema project is valid and the main Protobuf message was validated by the Maven plugins. The following proto/target/proto/META-INF/validation-results.json file should be generated:

{
  "passedValidators" : [ "GoogleStyle", "FileExtension", "MajorVersionInPackage", "BackwardsCompatibility" ]
}

If your schema project has an invalid main message file, the olp schema package command should return the following output with a path to the log file where you can find additional information about what went wrong with your project:

ERROR: Schema packaging failed.
Log file path: /tmp/olpcli2202976880595062312.log

To disable validators in the schema project, you can set the googleStyle, fileExtension, backwardsCompatibility, majorVersionInPackage, packageConsistency configurations to false in the proto/pom.xml and ds/pom.xml files:

<plugin>
  <groupId>com.here.platform.schema.maven_plugins</groupId>
  <artifactId>proto-validate-plugin</artifactId>
  <version>${here.plugin.version}</version>
  <executions>
    <execution>
      <id>validate</id>
      <phase>compile</phase>
      <goals>
        <goal>validate</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <googleStyle>false</googleStyle>
    <fileExtension>false</fileExtension>
    <backwardsCompatibility>false</backwardsCompatibility>
    <majorVersionInPackage>false</majorVersionInPackage>
    <ignorePackages>
      <ignorePackage>google.protobuf</ignorePackage>
    </ignorePackages>
  </configuration>
</plugin>

Note

It is not recommended to disable plugins, as this may cause schemes to malfunction.

Once the project is validated and packaged, we can publish it to the platform.

Publish schema to the platform

Let's deploy the Protobuf schema to the platform, using the olp schema put OLP CLI command:

olp schema put --input /home/user/generate_protobuf_schema

The OLP CLI should return the following output:


WARNING: Please wait while the command is completed. This may take several minutes.

Schema hrn:here:schema::olp-here:com.here.tutorial:generate_protobuf_schema_v1:1.0.0 has been created.

Please use 'olp schema show' command to get more information.

Once the schema is deployed, you can see its details in the OLP CLI or portal.

Get details about published schema

Let's get more information about the deployed schema using the olp schema show command:


olp schema show hrn:here:schema::olp-here:com.here.tutorial:generate_protobuf_schema_v1:1.0.0

The OLP CLI should return the following information:


Details of the Schema schema:
schema version           1.0.0
artifact id              generate_protobuf_schema_v1
group id                 com.here.platform.tutorial
type                     proto
summary                  Schema project for OLP

Schema validation results:
module         backwards compatibility       file extension      google style        major version in package      package consistency
proto          true                          true                true                true                          false
ds             false                         false               false               false                         true

Artifacts:
hrn:here:artifact::olp-here:com.here.platform.tutorial:generate_protobuf_schema_v1_java:1.0.0
hrn:here:artifact::olp-here:com.here.platform.tutorial:generate_protobuf_schema_v1_proto:1.0.0
hrn:here:artifact::olp-here:com.here.platform.tutorial:generate_protobuf_schema_v1_scala_2.12:1.0.0
hrn:here:artifact::olp-here:com.here.platform.tutorial:generate_protobuf_schema_v1_ds:1.0.0

From the output above, we get general information about the deployed schema, particularly, schema version, group id, artifact id, schema type, and schema summary. Schema validation results indicate which plugins were used for which modules. The last section of the output contains details of the artifacts.

When you deploy a Protobuf schema, four artifacts are deployed as well: _proto, _ds, _java, _scala_2.12.

The _proto artifact contains a zip file containing the Protobuf message source. The artifact contains the main Protobuf file and validation results in JSON format. The optional file is description.md. This file contains documentation.

Note

The documentation.md file can be overridden by the docURL property in the schema.yml file. The docURL has a higher priority, so if both documentation.md and docUrl are specified, the docUrl will be used.

To override the documentation.md file when generating a schema, use the --doc-url <URL to the documentation> parameter with the olp schema generate command.

The _ds artifact contains a zip file containing a data schema (ds) bundle with the Protobuf messages. To download the _ds artifact, see the Download schema section.

The _java artifact contains a jar file containing Protobuf bindings for the Java language.

The _scala_2.12 artifact contains a jar file containing Protobuf bindings for the Scala 2.12 language.

Download schema

To download a _ds package with Protobuf messages, the OLP CLI provides the olp schema get command.


olp schema get hrn:here:schema::olp-here:com.here.tutorial:generate_protobuf_schema_v1:1.0.0 --output ~/Desktop/ds_package

The OLP CLI should return the following output:

Schema package was successfully downloaded to ~/Desktop/ds_package/generate_protobuf_schema_v1_ds-1.0.0.zip

The downloaded zip file contains the following structure:

generate_protobuf_schema_v1_ds-1.0.0/
├── com
│   └── here
│       └── platform
│           └── tutorial
│               └── generate_protobuf_schema
│                   └── v1
│                       └── generate_protobuf_schema.proto
├── google
│   └── protobuf
│       ├── any.proto
│       ├── api.proto
│       ├── compiler
│       │   └── plugin.proto
│       ├── descriptor.proto
│       ├── duration.proto
│       ├── empty.proto
│       ├── field_mask.proto
│       ├── source_context.proto
│       ├── struct.proto
│       ├── timestamp.proto
│       ├── type.proto
│       └── wrappers.proto
├── META-INF
│   ├── layer.fds
│   ├── layer.manifest.json
│   └── validation-results.json
├── renderers
│   └── ReadMe.txt
└── ResourcesReadMe.txt

From the project structure above, we can see that the _ds package contains the main generate_protobuf_schema.proto message, validation-results.json validation results, and a layer.manifest.json manifest that declares the top-level message. Also, the _ds module contains the com.google.protobuf package with declared Protocol Buffers Well-Known Types.

Use Maven archetype

There is an additional option to generate, validate, and upload schemas. Since a Protobuf schema project is a Maven project, you can use Maven commands to work with schemas. For this purpose, the Data SDK for Java & Scala provides the project_archetype that allows you to generate a Protobuf schema project using Maven.

Note

In order to execute the commands listed below, you need to have Maven installed on your local machine.

To generate a Protobuf schema project with the project_archetype, use the following Maven command that does the same as the olp schema generate com.here.platform.tutorial generate_protobuf_schema command in the Generate Protobuf schema section:

Linux
Windows
mvn archetype:generate -DarchetypeGroupId=com.here.platform.schema \
-DarchetypeArtifactId=project_archetype \
-DarchetypeVersion=2.1.106 \
-DartifactId=generate_protobuf_schema \
-Dversion=1.0.0 \
-DgroupId=com.here.platform.tutorial \
-DmajorVersion=1
mvn archetype:generate -DarchetypeGroupId=com.here.platform.schema ^
-DarchetypeArtifactId=project_archetype ^
-DarchetypeVersion=2.1.106 ^
-DartifactId=generate_protobuf_schema ^
-Dversion=1.0.0 ^
-DgroupId=com.here.platform.tutorial ^
-DmajorVersion=1

To validate and build the generated schema, use the mvn install command from the root of the project which is similar to the olp schema package command.

To deploy the schema without using the olp schema put command, you can run the mvn deploy command from the root of the generated project.

Once you have deployed the schema, you can inspect it using the OLP CLI commands mentioned in this tutorial.

Best practices

This chapter describes some of the best practices of creating and organizing your schemas. Following these guidelines helps you improve your HERE platform experience in general and minimize a number of errors when you upload your schema to the platform or install it locally.

Schema naming

Try to limit the names of your schemas to 30 characters. This way, your schema name fits in one line and is easy to read in the HERE platform portal.

You can specify the name of your schema in the schema project POM file: generate_protobuf_schema/pom.xml

Schema descriptions

In a schema description, include the purpose of your schema. Also, if your schema is dependent on other schemas, try to list these dependencies in the description as well.

To keep your schema descriptions neat and readable, limit them to 250 characters. This translates into 3 lines of text in the schemas list (HERE platform portal > Data > Browse schemas).

Package naming

Group ID and artifact ID are used to generate the schema's HERE Resource Name (HRN) once you upload your schema to the platform.

The package name is used in the Protobuf, Java, and Scala package names.

For groupId, use your company's reversed domain name. Besides, you should add a team/product hierarchy to the groupId, for example, com.some-company.automated-driving.schema.

Note

When a schema is being deployed to the Artifact Service, its groupId is being reserved by the organization of the schema owner. Users from other organizations will not be able to deploy their schemas if they share the same groupId. Therefore, using generic groupId values like com.example, com.here.schema, com.here.example may make it impossible to upload the schema to the Artifact Service repository and thus using it in catalogs or sharing it with other users.

For artifact ID, use a name specific to the type of schemas packaged in the project, such as building-footprints. Try to keep the artifact ID as short as possible.

We recommend following Java package naming conventions.

Versioning

Use the semantic versioning system (SemVer 2.0.0) to designate the version of your schema.

File naming

Make sure that all your Protobuf files in the schema have the .proto extension.

For naming .proto files, use the snake_case style. building_footprints_partition.proto and locations_partition.proto are good examples of how proper Protobuf file names should look.

Package structuring

Make sure that the directory tree of the Protobuf source files corresponds to the Protobuf package name in the individual .proto files. For example, the proto/src/main/proto/com/here/example/v1/test_schema.proto file must contain the following package declaration: com.here.example.v1.

If your schema uses Protobuf files from other schemas, make sure that the import statement in an individual .proto file contains the path to the imported resource in the following format:

package name declaration + imported .proto file name, separated by forward slashes

Code styling

Use the Google's Proto3 Language Guide for your protobuf code. And follow the Google's Protobuf Style Guide for the names of messages, fields, and enums in your code.

Conclusion

In this tutorial, you have practiced generating, building, and publishing a Protobuf schema to the platform using the OLP CLI and Maven and got acquainted with the Protobuf schema structure and the OLP CLI commands that allow you to work with Protobuf schemas.

Further information

results matching ""

    No results matching ""