Expand Collapse
Just a second...

Using missing topic notifications with fan-out

Missing topic notifications generated by subscription or fetch requests to a secondary server are propagated to missing topic handlers registered against the primary servers.

Control client sessions can use missing topic notifications to monitor the activity of end-user client sessions. In response to subscription or fetch requests to missing topics, the control client session can choose to take an action, such as creating the missing topic.

For more information, see Handling subscriptions to missing topics.

How notification propagation works

A missing topic notification is propagated from a secondary server to a missing topic handler registered against a primary server if and only if all of the following conditions are met:
  • There is a session between the secondary server and the primary server.
  • The selector used for the subscription or fetch request to the secondary server intersects with one or more of the fan-out links to the primary server that are configured at the secondary server.
  • On the secondary server, there are no currently replicated topics that match both the fan-out link and selector used in the subscription or fetch request.
  • The primary server has no topics that match the selector used in the subscription or fetch request.
  • One or more missing topic handlers are registered against the primary server for a path that matches the selector. The following rules are used to select which missing topic handler receives the notification:
    • If multiple handlers are registered for the branch, the handler for the most specific topic path is notified.
    • If there is more than one handler for a path, the Diffusion™ server notifies a single handler.

The handler can use the supplied callback to respond proceed or cancel. The subscription or fetch operation is delayed until the handler responds, and is abandoned if the response is cancel.

Example flow

Figure 1. Missing topic notification propagation An architecture comprising a control client that connects to the primary server; a primary server with a topic tree containing topics A, A/B, and A/C; a secondary server that uses fan-out replication to replicate the a branch of the topic tree from the primary server; and an end-user client.
  1. A control client connects to the primary server and registers a missing topic notification handler against the A branch of the topic tree.
  2. A secondary server connects to the primary server and replicates the A branch of the topic tree.
  3. On the secondary server the replicated part of the topic tree comprises the following topics: A, A/B and A/C.
  4. An end-user client attempts to subscribe to A/D, which does not exist.
  5. The topic A/D is in part of the topic tree that is matched by a fan-out link selector, so the secondary server propagates the missing topic notification to the primary server.
  6. The topic A/D does not exist on the primary server, so the primary server sends the missing topic notification to the handler registered by the control client.

Missing topic notification handlers at both the primary and secondary servers

A single subscription or fetch can cause a missing topic notification to be sent to a handler registered against the secondary server as well as a handler registered against a primary server.

The decision about whether to notify the handlers registered against a primary server is based on the intersection of the selector used by the subscription or fetch with the selector used to configure the fan-out link. It is possible for a missing topic notification to be sent to the primary server, but not to local handlers because the selector matches other (non-replicated) topics hosted by the secondary.

In particularly complex configurations, multiple primary servers might receive the notification or there can be multiple tiers of fan-out connections.

Where multiple handlers are notified, the subscription or fetch operation is delayed until the all handlers respond, and the operation is abandoned if any response is cancel.

Considerations when using missing topic notifications with fan-out

Missing topic notifications are only propagated if both the primary and secondary server are Diffusion version 5.9.1 or later.

The intersection of the selector used by the subscription or fetch request with a selector used for a fan-out link is calculated based only on the path-prefix of each selector. Complex selectors that use regular expressions can produce false positive results or false negative results. We recommend that you do not use regular expressions in the selectors used to configure fan-out links.

Ensure that the principal that the secondary server uses to make the fan-out connection to the primary server has the SELECT_TOPIC permission for the path prefix of the selector that triggered the missing topic notification.

A current session must exist between the secondary server and the primary server to forward notifications. If there is no session or the session fails while the missing topic notification is in-flight, the secondary server logs a warning message and discards the notification. The subscription or fetch operation is completed as if the primary handler had responded proceed.

The robustness of the session between the servers can be improved by configuring reconnection. Fan-out connections can have a large number of messages in flight. It might be necessary to tune the reconnection time-out and increase queue depth and recovery buffer sizes.

This page last modified: 2016/10/03