Interface ITopicViews
This feature allows a client session to manage topic views.
Inherited Members
Namespace: PushTechnology.ClientInterface.Client.Features
Assembly: Diffusion.Client.dll
Syntax
public interface ITopicViews : IFeature
Remarks
A topic view maps one part of a server's topic tree to another. It dynamically creates a set of reference topics from a set of source topics, based on a declarative topic view specification. The capabilities of topic views range from simple mirroring of topics within the topic tree to advanced capabilities including publication of partial values, expanding a single topic value into many topics, throttling the rate of publication, and applying a fixed delay to the publication.
A topic view can also map topics from another server (in a different cluster). This capability is referred to as 'remote topic views'. The view can specify the server that the source topics are hosted on in terms of a remote server (see IRemoteServers for details of how to create and maintain remote servers).
Each reference topic has a single source topic and has the same topic type as its source topic. Reference topics are read-only (they cannot be updated), nor can they be created or removed directly. Otherwise, they behave just like standard topics. A client session can subscribe to a reference topic, and can fetch the reference topic's current value if it has one.
The source topics of a topic view are defined by a topic selector. One or more reference topics are created for each source topic, according to the topic view. If a source topic is removed, reference topics that are derived from it will automatically be removed. If a topic is added that matches the source topic selector of a topic view, corresponding reference topics will be created. Removing a topic view will remove all of its reference topics.
Topic view specifications
The following is a simple topic view specification
that mirrors all topics below the path a
to reference topics below the path b
.
map ?a// to b/<path(1)>
A topic view with this specification
will map a source topic at the path a/x/y/z
to a reference topic at the path b/x/y/z
.
The specification is simple,
so the reference topic will exactly mirror the source topic.
A general topic view specification comprises several parts:
- The source topic clause identifies the source topics.
- The path mapping clause determines how reference topic paths are derived from the source topic paths, and when expanding to more than one reference topic, from where the values are obtained.
- The optional topic property mapping clause determines how reference topic properties are derived from source topic properties.
- The optional value mapping clause determines how reference topic values are derived from source topic or expanded values.
- The optional throttle clause constrains the rate at which each reference topic is updated when its source topic is updated.
- The optional delay by clause causes a change to a view's source topic to be delayed by a fixed time before it is reflected in reference topics.
Source topic clause
The source topic clause begins with the
map
keyword
and is followed by a topic selector.
These topic selectors follow the same parsing rules
as other topic selectors.
When evaluating a topic view, topics in the topic tree that match the source topic selector are considered, with the following exceptions:
- Topics created through the Publisher API;
- ROUTING topics.
Both SLAVE and reference topics are valid source topics. In particular, chaining of topic views is supported; that is, a reference topic created by one topic view can be the source topic of another topic view. Additionally, a reference topic can be the master topic of a slave topic, or the source topic of a routing topic subscription.
Tip
Prefer topic views to slave topics. Individual topics can be mirrored by creating a slave topic, but maintaining slave topics for a branch of the topic tree quickly becomes tedious. A topic view will maintain such a branch automatically, and provides more sophisticated mapping options.
From clause
The from
clause optionally follows the source topic clause. It begins with
the from
keyword and is followed by a remote server name. The name
refers to a remote server created using the IRemoteServers feature.
The presence of the clause indicates that the source topics will be selected from the specified server and not from the local server.
Further details regarding the processing of remote topic views are given below.
Path mapping clause
The paths of reference topics are derived from the source topic according to the topic view path mapping. The path mapping allows the source topic path and the value of the source topic to determine the path of the reference topic. In addition the path mapping can include expand directives which allow objects and arrays in JSON source topic values to be expanded to produce many reference topics.
A path mapping clause begins with the
to
keyword
and is followed by a path mapping template.
A path mapping template is a topic path
with embedded directives.
Directives are evaluated when creating the topic reference
and substituted into the topic path.
Directives are delimited by angle brackets
<
,
>
and consist of the name of the directive
and a list of parameters.
The parameter list is comma-separated
and surrounded by parentheses ((
, )
).
The following mapping directives are supported:
Source path directives extract a portion of the source path
and are parameterized by the index
of the start part of the source path
and the number of parts to include.
The number of parts parameter is optional
– if it is missing,
the selection extends to the end of the source path.
The syntax is
<path(start, number)>
,
or <path(start)>
when the number of parts parameter is omitted.
For example,
given the source path
a/b/c/d
,
the source path directive
<path(1, 2)>
is mapped to the reference topic path
b/c
,
and the source path directive
<path(2)>
is mapped to the reference topic path
c/d
.
Topic property mapping clause
The ITopicSpecification of a reference topic is derived from the topic specification of the source topics. A reference topic has the same topic type as its source topic.
The topic properties of a reference topic are derived from the source topic. Some topic properties can be tuned using the optional topic property mapping clause. The following table describes the behavior for each topic property.
Source topic property | Reference topic specification default | Can be set by topic property mapping? | Notes |
---|---|---|---|
Compression | Copied from source topic specification | Yes | |
Conflation | Copied from source topic specification | Yes | |
DontRetainValue | Copied from source topic specification | Yes | |
Owner | Not set | No | |
Persistent | Not set | No | Reference topics are not persisted. Topic views are persisted, so a reference topic will be recreated on server restart if its source is persistent. |
Priority | Copied from source topic specification | Yes | |
PublishValuesOnly | Copied from source topic specification | Yes | |
Removal | Not set | No | Reference topics cannot be removed directly. |
Schema | Copied from source topic specification | No | A RECORD_V2 reference topic has the same schema as its source topic. |
SlaveMasterTopic | Not set | No | If a reference topic has a slave topic as its source topic, it indirectly references the slave's master topic. |
TidyOnUnsubscribe | Copied from source topic specification | Yes | |
TimeSeriesEventValueType | Copied from source topic specification | No | A TIME_SERIES reference topic has the same value type as its source topic. |
TimeSeriesRetainedRange | Copied from source topic specification | Yes, with restrictions | A topic property mapping cannot increase the time series retained range by overriding the TimeSeriesRetainedRange property. The retained range of a reference time series topic will be constrained to be no greater than that of its source topic. |
TimeSeriesSubscriptionRange | Copied from source topic specification | Yes | |
ValidateValues | Not set | No | A reference topic reflects updates to its source topic. It cannot reject updates. |
A topic property mapping clause begins with the keywords
with properties
and consists of a comma-separated list
of topic property keys and values,
each separated by a colon.
For example,
the following topic view specification
maps all topics below the path
a
to reference topics below the path
b
,
and disables both conflation
and compression for the reference topics.
map ?a// to b/<path(1)> with properties 'CONFLATION':'off', 'COMPRESSION':'false'
Topic value mapping
By default, a reference topic's value is a copy of the source topic value, or part of the source value produced by an expand path mapping directive. For JSON source topics, the optional topic value mapping clause can be applied to extract part of the source value, or to further refine the value produced by the expand directive.
A topic value mapping begins with the keyword
as
and is followed by a value directive.
A value directive is delimited by angle brackets
(<
, >
),
and consists of the
value
keywords and a single JSON pointer parameter.
The JSON pointer selects the part of the source value to copy.
For example, given a source value of
{
"account" : "1234",
"balance" : { "amount" : 12.57, "currency" : "USD" }
}
and the value mapping clause
as <value(/balance)>
,
the reference topic value will be
{
"amount" : 12.57,
"currency" : "USD"
}
Value mappings that follow expand directives apply to the expanded value and not the source topic value.
Topic value mappings only alter the reference topic value;
only the path mapping determines whether a reference topic
should exist. If the topic value mapping's JSON pointer fails
to select anything from the source topic value, the reference
topic will have the JSON value null
.
Topic value mappings are often used with path value mappings to avoid repeating information in the path and the value. For example:
map ?accounts// to balances/<scalar(/account)> as <value(/balance)>
Value mappings that follow expand directives apply to the expanded value and not the source topic value.
Throttle clause
The optional throttle clause can be used to constrain the rate at which a reference topic is updated when its source topic is updated. The primary application of a throttle clause is to restrict the number of updates sent to reference topic subscribers, reducing network utilization or the processing each subscriber must do. Throttling also restricts the rate at which client sessions can observe changes to reference topic values using the fetch API.
The throttle clause has the form
throttle to X updates every period
,
where X is a positive integer,
and period is a positive integer
followed by a time unit which is one of
seconds
,
minutes
,
or hours
.
For example,
the following topic view specification
maps all topics below the path
a
to reference topics below the path
b
,
but updates the value of each reference topic
at most twice every five seconds:
map ?a// to b/<path(1)> throttle to 2 updates every 5 seconds
To improve readability,
the throttling clause allows
1 update
as an alternative to
1 updates
,
and every second
as an alternative to
every 1 seconds
(and so on, for other time units).
For example,
the following topic view specification
maps all topics below the path
a
to reference topics below the path
b
,
but updates the value of each reference topic
at most once every hour:
map ?a// to b/<path(1)> throttle to 1 update every minute
The throttle clause is only applied when a source topic is updated more frequently than the configured rate. If a source topic is updated less frequently, updates are passed on unconstrained. If the rate is exceeded, a reference topic will not be updated again until the configured period has expired. At this time, the reference topic will be updated based on the source topic updates that happened in the interim, and a single value will be published. Thus, a throttle clause provides topic-scoped conflation.
The throttle clause is ignored for time series topics because time series updates do not support efficient conflation. Updates to source time series topics are passed on immediately to the corresponding reference topics, regardless of any throttle clause.
Delay clause
The optional delay clause causes a change to a view's source topic to be delayed by a fixed time before it is reflected in reference topics. Topic additions, updates, and removals are all delayed. Delays can range from one second to many days.
Such a publication delay is a useful way to devalue topic data so it can be given away to non-paying users.
The delay clause has the form
delay by duration
, where
duration is a positive integer followed by a time unit which is one
of seconds
, minutes
, or hours
.
For example, the following topic view specification maps all topics below the path
a
to reference topics below the path b
, but
changes to a source topic are delayed by five minutes before they are
reflected in the corresponding reference topic.
map? a// to b/<path(1)> delay by 5 minutes
Views with delay clauses initially create reference topics in an unpublished state. The topics are published once the delay time has expired. A topic in the unpublished state prevents a lower priority topic view from creating a reference topic with the same path. Sessions with the rights to read the source topic can browse unpublished topics using the WithUnpublishedDelayedTopics fetch request option.
Escaping and quoting special characters
Each part of a topic view expression
has characters with special significance.
Source topic clauses
and path mapping clauses
are delimited by white space.
Directives in path
and topic property mapping clauses
are delimited by the
<
and >
characters,
and each directive parameter is terminated by
,
or
)
.
Topic property mapping
clauses are delimited by white space,
and the :
and ,
characters.
Sometimes a topic view must refer to
or generate topics with paths
that contains special characters,
or use a JSON pointer containing special characters.
The escape sequence
\x
can be used to literally insert any character
x
,
with a one exception:
\/
cannot be used in path fragments
since the path delimiter
/
is always significant.
Here is an example topic view expression
containing escape sequences.
It maps the topic path
a topic
a reference topic with the path
another topic
.
map a\ topic to another\ topic
Here is an example with a source value directive
that uses the JSON pointer
/x()/y
to extract the target path from the source value.
The
)
character in the JSON pointer must be escaped
so it is not treated as the end of the parameter list.
map ?a// to <scalar(/x(\)/y)>
To insert
\
,
the escape sequence
\\
must be used.
There is no need to escape white space
in JSON pointers directive parameters.
However,
white space is significant.
For example,
the following expressions have different topic value mapping clauses
since the JSON pointer in the second expression is
"/x "
;
that is,
it has a trailing space:
map a to b as <value(/x)>
map a to b as <value(/x )>
Instead of using escape sequences,
white space characters can be included in source topic clauses
and path mapping clauses using quotes.
A clause is quoted by wrapping it in single quote
('
)
or double quote
("
)
characters.
For example:
map "a topic" to "another topic"
Within a quoted clause, quotes of the same type must be escaped:
map 'alice\'s topic' to 'bob\'s topic'
For consistency, the values in topic property mapping clauses can be escaped or quoted. However, there is no need to do so because none of the valid values for the mappable properties contain special characters.
Dealing with topic path conflicts
Reference topics have a lower priority than normal topics created through the API, including replicas of normal topics created by topic replication or fan-out. A reference topic will only be created if no topic or reference topic is already bound to its derived topic path.
Topic views have a precedence based on order of creation. If two topic views define mappings the same topic path, the earliest-created topic view will create a reference topic. If a topic view is updated, it retains its original precedence.
Remote topic views
A remote topic view is one that specifies another server as the location of the source topics using the from clause as shown in the example below:
map? a// from server1 to b/<path(1)>
The server name server1
in this example refers to the name of a
remote server created using the IRemoteServers feature.
In this case, upon establishing a successful connection with the remote server indicated the topic view will create reference topics locally based upon the topics selected by the topic view's selector at the remote server. It is important to note that the selector only refers to topics that match it at the remote server and not on the local server and there is no reason why there could not be a source topic at the remote server that has the same path as an entirely different topic on the local server.
More than one topic view can specify the same remote server.
A remote server only makes a physical connection when it is in use, therefore the first topic view that specifies a remote server will cause it to establish a connection. Similarly, if the last topic view that uses a remote server is removed then the connection will be closed.
It is not necessary for the named remote server definition to exist before creating the topic view, as if it does not then the topic view will simply remain dormant until the remote server is created and a successful connection to the server specified in its URL is established. Similarly, if a remote server that is in use by remote topic views is removed then all of the reference topics created by the topic views will be removed and the topic views will become dormant until the named remote server is created again or the views are changed to name a different remote server.
If a remote topic view selects a SLAVE
topic at the
remote server then local mappings will only be performed when the slave topic
binds to another topic and reference topics would be removed if the slave
topic becomes unbound (i.e. its master topic is removed). If a remote topic
view selects a ROUTING
topic at the remote server
then local mappings will only be performed if the routing topic mapping at
the remote server is able to establish a mapping for the remote server
connection. In both cases the mapping will be done as if from the resolved
slave or routing topic.
The rules of precedence for remote topic views are the same as for other topic views. If the remote server for a remote topic view does not exist or does not have an established connection then the remote topic view is not evaluated (i.e. it is as if the source topics for the view did not exist), but if the remote server later connects then the view will be evaluated and rules of precedence will determine whether reference topic will replace those created by earlier views.
Topic view persistence and replication
Reference topics are neither replicated nor persisted. They are created and removed based on their source topics. However, topic views are replicated and persisted. A server that restarts will restore topic views during recovery. Each topic view will then create reference topics based on the source topics that have been recovered.
The server records all changes to topic views in a persistent store. Topic views are restored if the server is started.
If a server belongs to a cluster, topic views (and remote servers) will be replicated to each server in the cluster. Topic views are evaluated locally within a server. Replicated topic views that select non-replicated source topics can create different reference topics on each server in the cluster. When remote topic views are in use, each server in the cluster will make a connection to the specified remote server and will separately manage their remote topic views.
A view with a delay clause uses temporary storage to record delayed events. If there is a high volume of updates, temporary per-server disk files will be used to save server memory. The storage is per-server, and does not survive server restart. When a server is started, no data will be published by a view with a delay clause until the delay time has expired.
Access control
The following access control restrictions are applied:
- To list the topic views, a session needs the READ_TOPIC_VIEWS global permission.
- To create, replace, or remove a topic view, a session needs the MODIFY_TOPIC_VIEWS global permission and SELECT_TOPIC permission for the path prefix of the source topic selector.
- Each topic view records the principal and security roles of the session that created it as the topic view security context. When a topic view is evaluated, this security context is used to constrain the creation of reference topics. A reference topic will only be created if the security context has READ_TOPIC permission for the source topic path, and MODIFY_TOPIC permission for the reference topic path. The topic view security context is copied from the creating session at the time the topic view is created or replaced, and is persisted with the topic view. The topic view security context is not updated if the roles associated with the session are changed.
Note
This interface does not require user implementation and is only used to hide implementation details.
Added in 6.3.
Examples
An instance of this feature can be obtained by calling TopicViews.
// session is a previously obtained ISession instance.
var topicViews = session.TopicViews;
Or it can be obtained by calling Topics.
// session is a previously obtained ISession instance.
var topics = session.Topics;
Methods
CreateTopicViewAsync(String, String)
Create a new named topic view.
Declaration
Task<ITopicView> CreateTopicViewAsync(string name, string specification)
Parameters
Type | Name | Description |
---|---|---|
String | name | The name of the view. |
String | specification | The specification of the topic view. |
Returns
Type | Description |
---|---|
Task<ITopicView> | The |
Remarks
If a view with the same name already exists the new view will update the existing view.
If the operation completes successfully, the Task
result will be a
ITopicView instance.
Exceptions
Type | Condition |
---|---|
InvalidTopicViewException | The supplied specification is invalid. Thrown by the returned |
SessionSecurityException | The calling session does not have MODIFY_TOPIC_VIEWS permission or
appropriate path prefix permissions. Thrown by the returned |
SessionClosedException | The calling session is closed. Thrown by the returned |
ClusterRoutingException | A transient cluster error occured. Thrown by the returned |
See Also
CreateTopicViewAsync(String, String, CancellationToken)
Create a new named topic view.
Declaration
Task<ITopicView> CreateTopicViewAsync(string name, string specification, CancellationToken cancellationToken)
Parameters
Type | Name | Description |
---|---|---|
String | name | The name of the view. |
String | specification | The specification of the topic view. |
CancellationToken | cancellationToken | The cancellation token used to cancel the current operation. |
Returns
Type | Description |
---|---|
Task<ITopicView> | The |
Remarks
The topic view name cannot be empty.
If a view with the same name already exists the new view will update the existing view.
If the operation completes successfully, the Task
result will be a
ITopicView instance.
Exceptions
Type | Condition |
---|---|
InvalidTopicViewException | The supplied specification is invalid. Thrown by the returned |
SessionSecurityException | The calling session does not have MODIFY_TOPIC_VIEWS permission or
appropriate path prefix permissions. Thrown by the returned |
SessionClosedException | The calling session is closed. Thrown by the returned |
ClusterRoutingException | A transient cluster error occured. Thrown by the returned |
ListTopicViewsAsync()
List all of the topic views that have been created.
Declaration
Task<IReadOnlyCollection<ITopicView>> ListTopicViewsAsync()
Returns
Type | Description |
---|---|
Task<IReadOnlyCollection<ITopicView>> | The |
Remarks
If the operation completes successfully, the Task
result will be a
Exceptions
Type | Condition |
---|---|
SessionSecurityException | The calling session does not have READ_TOPIC_VIEWS permission.
Thrown by the returned |
SessionClosedException | The calling session is closed. Thrown by the returned |
See Also
ListTopicViewsAsync(CancellationToken)
List all of the topic views that have been created.
Declaration
Task<IReadOnlyCollection<ITopicView>> ListTopicViewsAsync(CancellationToken cancellationToken)
Parameters
Type | Name | Description |
---|---|---|
CancellationToken | cancellationToken | The cancellation token used to cancel the current operation. |
Returns
Type | Description |
---|---|
Task<IReadOnlyCollection<ITopicView>> | The |
Remarks
If the operation completes successfully, the Task
result will be a
Exceptions
Type | Condition |
---|---|
SessionSecurityException | The calling session does not have READ_TOPIC_VIEWS permission.
Thrown by the returned |
SessionClosedException | The calling session is closed. Thrown by the returned |
RemoveTopicViewAsync(String)
Remove a named topic view if it exists.
Declaration
Task<object> RemoveTopicViewAsync(string name)
Parameters
Type | Name | Description |
---|---|---|
String | name | The name of the view. |
Returns
Type | Description |
---|---|
Task<Object> | The |
Remarks
If the named view does not exist the Task
will complete successfully.
If the task completes successfully, the Task
result will be null
.
Exceptions
Type | Condition |
---|---|
SessionSecurityException | The calling session does not have MODIFY_TOPIC_VIEWS permission.
Thrown by the returned |
SessionClosedException | The calling session is closed. Thrown by the returned |
ClusterRoutingException | A transient cluster error occured. Thrown by the returned |
See Also
RemoveTopicViewAsync(String, CancellationToken)
Remove a named topic view if it exists.
Declaration
Task<object> RemoveTopicViewAsync(string name, CancellationToken cancellationToken)
Parameters
Type | Name | Description |
---|---|---|
String | name | The name of the view. |
CancellationToken | cancellationToken | The cancellation token used to cancel the current operation. |
Returns
Type | Description |
---|---|
Task<Object> | The |
Remarks
If the named view does not exist the Task
will complete successfully.
If the task completes successfully, the Task
result will be null
.
Exceptions
Type | Condition |
---|---|
SessionSecurityException | The calling session does not have MODIFY_TOPIC_VIEWS permission.
Thrown by the returned |
SessionClosedException | The calling session is closed. Thrown by the returned |
ClusterRoutingException | A transient cluster error occured. Thrown by the returned |