Diffusion Apple API  6.6.2
Unified Client Library for iOS, tvOS and OS X / macOS
 All Classes Files Functions Variables Enumerations Enumerator Properties Pages
Instance Methods | List of all members
PTDiffusionTopicViewsFeature Class Reference

Introduction

This feature allows a client session to manage topic views.

A topic view maps one part of a server's topic tree to another. It dynamically creates a set of reference topics from a set of source topics, based on a declarative topic view specification. The capabilities of topic views range from simple mirroring of topics within the topic tree to advanced capabilities including publication of partial values, expanding a single topic value into many topics, inserting values from other topics, throttling the rate of publication, and applying a fixed delay to the publication.

A topic view can also map topics from another server (in a different cluster). This capability is referred to as 'remote topic views'. The view can specify the server that the source topics are hosted on in terms of a remote server.

Each reference topic has a single source topic and has the same topic type as its source topic. Reference topics are read-only (they cannot be updated), nor can they be created or removed directly. Otherwise, they behave just like standard topics. A client session can subscribe to a reference topic, and can fetch the reference topic's current value if it has one.

The source topics of a topic view are defined by a topic selector. One or more reference topics are created for each source topic, according to the topic view. If a source topic is removed, reference topics that are derived from it will automatically be removed. If a topic is added that matches the source topic selector of a topic view, corresponding reference topics will be created. Removing a topic view will remove all of its reference topics.

Topic view specifications

The following is a simple topic view specification that mirrors all topics below the path a to reference topics below the path b.

 map ?a// to b/<path(1)>
 

A topic view with this specification will map a source topic at the path a/x/y/z to a reference topic at the path b/x/y/z. The specification is simple, so the reference topic will exactly mirror the source topic.

A general topic view specification comprises several parts:

Source topic clause

The source topic clause begins with the map keyword and is followed by a topic selector. These topic selectors follow the same parsing rules as other topic selectors.

When evaluating a topic view, all topics in the topic tree that match the source topic selector are considered, excluding Routing topics.

Reference topics are valid source topics. In particular, chaining of topic views is supported; that is, a reference topic created by one topic view can be the source topic of another topic view. Additionally, a reference topic can be the source topic of a routing topic subscription.

From clause

The from clause optionally follows the source topic clause. It begins with the from keyword and is followed by a remote server name. The name refers to a remote server created using the PTDiffusionRemoteServersFeature feature.

The presence of the clause indicates that the source topics will be selected from the specified server and not from the local server.

Further details regarding the processing of remote topic views are given below.

Path mapping clause

The paths of reference topics are derived from the source topic according to the topic view path mapping. The path mapping allows the source topic path and the value of the source topic to determine the path of the reference topic. In addition the path mapping can include expand directives which allow objects and arrays in JSON source topic values to be expanded to produce many reference topics.

A path mapping clause begins with the to keyword and is followed by a path mapping template. A path mapping template is a topic path with embedded directives. Directives are evaluated when creating the topic reference and substituted into the topic path. Directives are delimited by angle brackets (<, >) and consist of the name of the directive and a list of parameters. The parameter list is comma-separated and surrounded by parentheses ((, )).

The following path mapping directives are supported:

Source path directives

Source path directives extract a portion of the source path and are parameterized by the index of the start part of the source path and the number of parts to include. The number of parts parameter is optional – if it is missing, the selection extends to the end of the source path. The syntax is <path(start, number)>, or <path(start)> when the number of parts parameter is omitted.

For example, given the source path a/b/c/d, the source path directive <path(1, 2)> is mapped to the reference topic path b/c, and the source path directive <path(2)> is mapped to the reference topic path c/d.

Source value ("scalar") directives

Source value directives are only applied to JSON source topics or Time Series source topics with a JSON event type; if the path mapping contains a source value directive, topics with other topic types matching the source topic selector are ignored. Source value directives use the keyword scalar and are parameterized by a single JSON pointer that extracts a scalar value from the source (or current) value. A scalar value is a string, a number, true, false, or null, that is, anything other than an array or an object. If the JSON pointer does not refer to a scalar value in the source (or current) value, no reference topic will be created. This includes cases where the JSON pointer refers to an array or an object), or when no part of the source value is selected.

Deriving the reference topic paths from part of the source topic value effectively creates a secondary index on the value. For source value directives to work efficiently, the selected scalar values should be relatively stable. If an update to the source topic changes the selected scalar value, the corresponding reference topic will be removed and a new reference topic will be created.

For example, given a source value of

 {
     "account" : "1234",
     "balance" : { "amount" : 12.57, "currency" : "USD" }
 }
 

and the source value directive currency/<scalar(/balance/currency)</account/<scalar(/account)>, the reference topic path will be currency/USD/account/1234.

If the extracted value is a string, it is copied literally to the reference topic path. A value that contains path separators (/) will create a reference topic path with more levels than the path mapping template. Use the separator directive to replace path separator with an alternative string.

An extracted value of null will be copied to the reference topic path as the string "null".

Expand value directives

Expand value directives are only applied to JSON source topics; if the path mapping contains an expand value directive, non-JSON topics matching the source topic selector are ignored.

Expand value directives use the keyword expand and are parameterized by one or two JSON pointers.

The first pointer indicates the element within the value to be expanded, and if omitted, the value is expanded from the root. Expansion of a source topic indicates that every direct child of the element pointed to by the expand pointer will be used to create a new reference topic (or provide input to later expand or scalar directives). For example <expand()> would expand every child item in the source value and <expand(/account)> would expand every child of the account value in the source value. The specified value could be an object, an array or even a scalar value, but a scalar value would expand to only a single new value.

The optional second parameter of the expand directive specifies a pointer to a scalar value within the expanded value which will be used to derive the path fragment of the reference topic path. If the second pointer is not specified or no scalar value is found for the pointer, the path fragment is taken from the key (if the child value is an object) or the index (if the child value is an array). Scalar child values will expand to a reference topic but will not add anything to the generated path. For example expand(,/name)> would expand from the root of the source value and each child value path fragment would be obtained from the scalar value with the key name.

So if a source topic had a value of

 {
 "values": [1, 5, 7]
 }
 

a path mapping of value<expand(/values)> would expand the value to the following reference topics:-

path value0 with a value of 1
path value1 with a value of 5
path value2 with a value of 7

Expand directives can be nested (i.e. there can be more than one expand directive in a path mapping). In this case a second expand directive will use the value from the previous expand as its source (root) value and not the value of the source topic. This also applies to scalar directives that follow an expand directive.

If expansion causes more than one mapping to the same topic path, only the first encountered will be created and updated.

Expanding source topic values effectively creates secondary indices on the value. For expanded value directives to work efficiently, the value selected for expansion should be relatively stable in terms of the children it contains. If an update to the source topic changes the children of the expanded value, then corresponding reference topics will be removed and created. Updates should generally be limited to changing values within the expanded values.

Insert clause(s)

Insert clauses may be specified directly after the path mapping, but before any other clauses, to indicate that values from other topics (insertion topics) are to be added to the derived reference topic value.

Insert clauses only apply to JSON source topics; if the topic view specification contains an insert clause, non-JSON topics matching the source topic selector are ignored.

The name of the topic to insert from can be specified in a similar way to the path mapping in that constants, path directives, and scalar directives (but not expand directives) may be used.

The value from the insertion topic (if found) is inserted into the 'current' value at a specified key position. The current value may be the source topic value, the value output from expand directives (in which case the insertion applies to each value), or the value from a previous insert clause. Insertion topics may be type JSON, String, Int64, or Double.

If, when evaluating a topic view, the insertion topic is not found (or the specified key within it is not found), or it is of an unsupported topic type, an optional default value may be inserted, but if no default is specified then no insertion takes place and the value is passed to the derived reference topic unchanged.

The values of the insertion topics are only taken at the point when the source topic is evaluated against the topic view (i.e. when the source topic is updated). Changes to the value of the insertion topic that occur in the interim are not applied to the derived reference topics.

The format of an insert clause is

 insert path [key fromKey] at insertionKey [default defaultValue]
 

The path is specified in exactly the same way as for the path mapping clause, except it may not contain expand directives. path directives operate on the path of the source topic, whereas scalar directives operate on the 'current' value as defined previously.

key is optional and can specify a fromKey which is a JSON pointer indicating the data within the insertion topic that is to be inserted. If no key is specified then the whole of the insertion topic value is inserted.

at specifies the insertionKey which is a JSON pointer indicating where to insert the insertion topic value. If a value already exists at the specified key then it is replaced otherwise it is inserted. Insertion can only occur if the parent of the key exists in the value and is of a compatible type. Array pointers may only be used to replace existing entries or append one greater than the last entry. The special pointer value /- may be used to append to the end of an existing array.

default is optional and may be used to specify a string defaultValue to be inserted if the insertion topic does not exist, it is of an unsupported topic type, or the specified key within it could not be found.

The following insert clause would cause the whole value of the topic named AnyTopic to be inserted into the current value at key /T, assuming that the current value is an object.

 insert AnyTopic at /T
 

The following insert clause would cause the whole value of the topic named AnyTopic to be inserted into the current value at key /T/MyKey, assuming that an object with the key T exists in the current value.

 insert AnyTopic at /T/MyKey
 

The following insert clause would cause the whole value of the topic named AnyTopic to be appended to the array at the key T in the current value.

 insert AnyTopic at /T/-
 

The following insert clause would cause the value at the key name within the topic named AnyTopic to be appended to the array at the key T in the current value.

 insert AnyTopic key /name at /T/-
 

In the above values no insertion would take place if the insertion value was not found, but a default value can be specified to insert into the current value in this case.

 insert AnyTopic at /T/- default "unknown"
 

The path of the topic to insert from can be built from parts of the source topic path and/or scalar values within the current value. For example:

 insert AC/<path(1,1)>/<scalar(/myval)> at /T
 

Any number of insert clauses may be chained together. For example:

 insert AnyTopic at /T/- default "unknown" insert AnotherTopic at /X/Another
 

Topic property mapping clause

The topic specification of a reference topic is derived from the topic specification of the source topics. A reference topic has the same topic type as its source topic.

The topic properties of a reference topic are derived from the source topic. Some topic properties can be tuned using the optional topic property mapping clause. The following table describes the behavior for each topic property.

Source topic property Reference topic specification default Can be set by topic property mapping? Notes
compressionPropertyKey Copied from source topic specification Yes
conflationPropertyKey Copied from source topic specification Yes
dontRetainValuePropertyKey Copied from source topic specification Yes
ownerPropertyKey Not set No
persistentPropertyKey Not set No Reference topics are not persisted. Topic views are persisted, so a reference topic will be recreated on server restart if its source is persistent.
priorityPropertyKey Copied from source topic specification Yes
publishValuesOnlyPropertyKey Copied from source topic specification Yes
removalPropertyKey Not set No Reference topics cannot be removed directly.
schemaPropertyKey Copied from source topic specification No A recordV2 reference topic has the same schema as its source topic.
tidyOnSubscribePropertyKey Copied from source topic specification Yes
timeSeriesEventValueTypePropertyKey Copied from source topic specification No A time series reference topic has the same value type as its source topic.
timeSeriesRetainedRangePropertyKey Copied from source topic specification Yes, with restrictions A topic property mapping cannot increase the time series retained range by overriding the TIME_SERIES_RETAINED_RANGE property. The retained range of a reference time series topic will be constrained to be no greater than that of its source topic.
timeSeriesSubscriptionRangePropertyKey Copied from source topic specification Yes
validateValuesPropertyKey Not set No A reference topic reflects updates to its source topic. It cannot reject updates.

A topic property mapping clause begins with the keywords with properties and consists of a comma-separated list of topic property keys and values, each separated by a colon. For example, the following topic view specification maps all topics below the path a to reference topics below the path b, and disables both conflation and compression for the reference topics.

 map ?a// to b/<path(1)> with properties 'CONFLATION':'off', 'COMPRESSION':'false'
 

Topic value mapping

By default, a reference topic's value is a copy of the source topic value, or part of the source value produced by an expand path mapping directive and/or modified by insert clauses. For JSON source topics or Time Series source topics with a JSON event type, the optional topic value mapping clause can be applied to extract part of the resulting value (the latest value in the case of time series topics).

A topic value mapping begins the keyword as and is followed by a value directive. A value directive is delimited by angle brackets (<, >), and consists of the value keywords and a single JSON pointer parameter. The JSON pointer selects the part of the current value to copy.

For example, given a current value of

 {
     "account" : "1234",
     "balance" : { "amount" : 12.57, "currency" : "USD" }
 }
 

and the value mapping clause as <value(/balance)>, the reference topic value will be

 {
     "amount" : 12.57,
     "currency" : "USD"
 }
 

Value mappings that follow expand directives and/or insert clauses apply to the current derived value and not the source topic value.

Topic value mappings only alter the reference topic value; only the path mapping determines whether a reference topic should exist. If the topic value mapping's JSON pointer fails to select anything from the source topic value, the reference topic will have the JSON value null.

Topic value mappings are often used with path value mappings to avoid repeating information in the path and the value. For example:

 map ?accounts// to balances/<scalar(/account)> as <value(/balance)>
 

Throttle clause

The optional throttle clause can be used to constrain the rate at which a reference topic is updated when its source topic is updated. The primary application of a throttle clause is to restrict the number of updates sent to reference topic subscribers, reducing network utilization or the processing each subscriber must do. Throttling also restricts the rate at which client sessions can observe changes to reference topic values using the fetch API. The throttle clause has the form:

 throttle to X updates every period
 

where X is a positive integer, and period is a positive integer followed by a time unit which is one of seconds, minutes, or hours. For example, the following topic view specification maps all topics below the path a to reference topics below the path b, but updates the value of each reference topic at most twice every five seconds:

 map ?a// to b/<path(1)> throttle to 2 updates every 5 seconds
 

To improve readability, the throttling clause allows 1 update as an alternative to 1 updates, and every second as an alternative to every 1 seconds (and so on, for other time units). For example, the following topic view specification maps all topics below the path a to reference topics below the path b, but updates the value of each reference topic at most once every hour:

 map ?a// to b/<path(1)> throttle to 1 update every minute
 

The throttle clause is only applied when a source topic is updated more frequently than the configured rate. If a source topic is updated less frequently, updates are passed on unconstrained. If the rate is exceeded, a reference topic will not be updated again until the configured period has expired. At this time, the reference topic will be updated based on the source topic updates that happened in the interim, and a single value will be published. Thus, a throttle clause provides topic-scoped conflation.

The throttle clause is ignored for time series topics because time series updates do not support efficient conflation. Updates to source time series topics are passed on immediately to the corresponding reference topics, regardless of any throttle clause.

Delay clause

The optional delay clause causes a change to a view's source topic to be delayed by a fixed time before it is reflected in reference topics. Topic additions, updates, and removals are all delayed. Delays can range from one second to many days​.

Such a publication delay is a useful way to devalue topic data so it can be given away to non-paying users​.

The delay clause has the form delay by duration, where duration is a positive integer followed by a time unit which is one of seconds, minutes, or hours.

For example, the following topic view specification maps all topics below the path a to reference topics below the path b, but changes to a source topic are delayed by five minutes before they are reflected in the corresponding reference topic.

 map ?a// to b/<path(1)> delay by 5 minutes
 

Views with delay clauses initially create reference topics in an unpublished state. The topics are published once the delay time has expired. A topic in the unpublished state prevents a lower priority topic view from creating a reference topic with the same path. Sessions with the rights to read the source topic can browse unpublished topics using the withUnpublishedDelayedTopics fetch request option.

Preserve topics clause

The default behavior for a topic view is that only the reference topics that can be derived from the current value of the source topic are maintained. This applies to views using directives that derive the path of the reference topic(s) from a value within the source topic (e.g. scalar or expand). For example if a view uses a scalar directive to determine the path of the reference topic and the source topic is updated such that the selected scalar value changes then the previously created reference topic will be removed and a new one created. Similarly an expand directive may create many reference topics from the source value but if the source is updated then only those reference topics that still have a corresponding value within the source value will be updated (or new ones added if not in the previous value), whilst others will be removed.

There are situations where an application may not want reference topics to be removed because of source value changes. In such cases the 'preserve topics' clause may be used to indicate that all reference topics created by a view should be retained (and updated by later updates that apply to it) until either the source topic or the creating view are removed.

For example:

 map ?a/path/ to b/<expand()> preserve topics
 

Separator clause

Views can use the scalar and expand directives in path mappings to extract text from the source value. By default, any / characters in the text are interpreted as path separators and will introduce extra levels in reference topic paths. If this is undesirable, the optional separator clause can be used to replace any / characters produced by the scalar and expand directives with a constant string. The replacement can contain path separators but must have no empty path segments (//).

Here is an example replacing all path separators with an alternate character:

 map ?a/path/ to b/<scalar(/x/y)> separator '%'
 

If the value at /x/y in the source topic was foo/bar, this would cause the reference topic to be created at /b/foo%bar instead of /b/foo/bar

Escaping and quoting special characters

Each part of a topic view expression has characters with special significance. Source topic clauses and path mapping clauses are delimited by white space. Directives in path and topic property mapping clauses are delimited by the < and > characters, and each directive parameter is terminated by , or ). Topic property mapping clauses are delimited by white space, and the : and , characters.

Sometimes a topic view must refer to or generate topics with paths that containing special characters, or use a JSON pointer containing special characters. The escape sequence \x can be used to literally insert any character x, with a one exception: \/ cannot be used in path fragments since the path delimiter / is always significant.

Here is an example topic view expression containing escape sequences. It maps the topic path a topic a reference topic with the path another topic.

 map a\ topic to another\ topic
 

Here is an example with a source value directive that uses the JSON pointer /x()/y to extract the target path from the source value. The ) character in the JSON pointer must be escaped so it is not treated as the end of the parameter list.

 map ?a// to <scalar(/x(\)/y)>
 

To insert \, the escape sequence \\ must be used.

There is no need to escape white space in JSON pointers directive parameters. However, white space is significant. For example, the following expressions have different topic value mapping clauses since the JSON pointer in the second expression is "/x "; that is, it has a trailing space:

 map a to b as <value(/x)>
 map a to b as <value(/x )>
 

Instead of using escape sequences, white space characters can be included in source topic clauses and path mapping clauses using quotes. A clause is quoted by wrapping it in single quote (') or double quote (") characters. For example:

 map "a topic" to "another topic"
 

Within a quoted clause, quotes of the same type must be escaped:

 map 'alice\'s topic' to 'bob\'s topic'
 

For consistency, the values in topic property mapping clauses can be escaped or quoted. However, there is no need to do so because none of the valid values for the mappable properties contain special characters.

Dealing with topic path conflicts

Reference topics have a lower priority than normal topics created through the API, including replicas of normal topics created by topic replication or fan-out. A reference topic will only be created if no topic or reference topic is already bound to its derived topic path.

Topic views have a precedence based on order of creation. If two topic views define mappings the same topic path, the earliest-created topic view will create a reference topic. If a topic view is updated, it retains its original precedence.

Remote topic views

A remote topic view is one that specifies another server as the location of the source topics using the from clause as shown in the example below:

 map ?a// from server1 to b/<path(1)>
 

The server name server1 in this example refers to the name of a remote server created.

In this case, upon establishing a successful connection with the remote server indicated the topic view will create reference topics locally based upon the topics selected by the topic view's selector at the remote server. It is important to note that the selector only refers to topics that match it at the remote server and not on the local server and there is no reason why there could not be a source topic at the remote server that has the same path as an entirely different topic on the local server.

More than one topic view can specify the same remote server.

A remote server only makes a physical connection when it is in use, therefore the first topic view that specifies a remote server will cause it to establish a connection. Similarly, if the last topic view that uses a remote server is removed then the connection will be closed.

It is not necessary for the named remote server definition to exist before creating the topic view, as if it does not then the topic view will simply remain dormant until the remote server is created and a successful connection to the server specified in its URL is established. Similarly, if a remote server that is in use by remote topic views is removed then all of the reference topics created by the topic views will be removed and the topic views will become dormant until the named remote server is created again or the views are changed to name a different remote server.

If a remote topic view selects a routing topic at the remote server then local mappings will only be performed if the routing topic mapping at the remote server is able to establish a mapping for the remote server connection. The mapping will be done as if from the resolved routing topic.

The rules of precedence for remote topic views are the same as for other topic views. If the remote server for a remote topic view does not exist or does not have an established connection then the remote topic view is not evaluated (i.e. it is as if the source topics for the view did not exist), but if the remote server later connects then the view will be evaluated and rules of precedence will determine whether reference topic will replace those created by earlier views.

Topic view persistence and replication

Reference topics are neither replicated nor persisted. They are created and removed based on their source topics. However, topic views are replicated and persisted. A server that restarts will restore topic views during recovery. Each topic view will then create reference topics based on the source topics that have been recovered.

The server records all changes to topic views in a persistent store. Topic views are restored if the server is started.

If a server belongs to a cluster, topic views (and remote servers) will be replicated to each server in the cluster. Topic views are evaluated locally within a server. Replicated topic views that select non-replicated source topics can create different reference topics on each server in the cluster. When remote topic views are in use, each server in the cluster will make a connection to the specified remote server and will separately manage their remote topic views.

A view with a delay clause uses temporary storage to record delayed events. If there is a high volume of updates, temporary per-server disk files will be used to save server memory​. The storage is per-server, and does not survive server restart. When a server is started, no data will be published by a view with a delay clause until the delay time has expired.

Access control

To list the topic views a session needs readTopicViews permission. To create or remove a topic view a session needs modifyTopicViews permission, selectTopic permission for the prefix of the source topic selector and modifyTopic permission for the prefix of the path mapping.

Accessing the feature

The Topic Views feature for a session can be obtained from the session's topicViews property.

See Also
PTDiffusionSession
Since
6.3
Inheritance diagram for PTDiffusionTopicViewsFeature:
PTDiffusionFeature

Instance Methods

(void) - createTopicViewWithName:specification:completionHandler:
 
(void) - listTopicViewsWithCompletionHandler:
 
(void) - removeTopicViewWithName:completionHandler:
 

Additional Inherited Members

- Properties inherited from PTDiffusionFeature
PTDiffusionSessionsession
 

Method Documentation

- (void) createTopicViewWithName: (NSString *)  name
specification: (NSString *)  specification
completionHandler: (PTDiffusionTopicView *_Nullable)  view
(NSError *_Nullable error)  completionHandler 

Create a new named topic view.

If a view with the same name already exists the new view will update the existing view.

Parameters
nameThe name of the view.
specificationThe specification of the view using the DSL.
completionHandlerBlock to be called asynchronously on success or failure. If the operation was successful, the error argument passed to the block will be nil. The completion handler will be called asynchronously on the main dispatch queue.
Exceptions
NSInvalidArgumentExceptionIf any argument is nil.
Since
6.3
- (void) listTopicViewsWithCompletionHandler: (NSArray< PTDiffusionTopicView * > *_Nullable)  views
(NSError *_Nullable error)  completionHandler 

List all the topic views that have been created.

Parameters
completionHandlerBlock to be called asynchronously on success or failure. If the operation was successful, the error argument passed to the block will be nil and views will contain a list of views sorted by their creation order. The completion handler will be called asynchronously on the main dispatch queue.
Exceptions
NSInvalidArgumentExceptionIf completionHandler is nil.
Since
6.3
- (void) removeTopicViewWithName: (NSString *)  name
completionHandler: (NSError *_Nullable error)  completionHandler 

Remove a named topic view if it exists.

If the named view does not exist then the operation is considered successful and the completion handler is called accordingly.

Parameters
nameThe name of the view.
completionHandlerBlock to be called asynchronously on success or failure. If the operation was successful, the error argument passed to the block will be nil. The completion handler will be called asynchronously on the main dispatch queue.
Exceptions
NSInvalidArgumentExceptionIf any argument is nil.
Since
6.3