Flink can read data continuously from non-stream layers. This means that during the runtime of the Flink execution environment the regular stream layer processing is ongoing and if there is a change in the data of a non-stream layer, the data is read and injected into the processing. This gives the functionality to not just read such layers once at the startup of the Flink execution environment but during runtime as well.
The behavior slightly differs depending on layer type like this:
For versioned layer this function is enabled by default. It uses layer notifications. Therefore it does not need to poll but gets notified if a new version is published. To disable this function just add the property
olp.connector-refresh-interval with value
-1 to your environment or application configuration.
Note: For versioned layer this feature works only for catalogs with
notificationsEnabled = true in catalog configuration.
Volatile and index layers:
For volatile and index layers you can define the refresh (polling) interval in milliseconds by adding the property
olp.connector-refresh-interval to your environment or application configuration. By default this interval is
60000L means 60 seconds.