Just a second...


Consider whether to use fan-out to replicate topic information from primary servers on one or more secondary servers.

A fan-out distribution comprises many servers that host the same topic or topics. When the topic is updated on a primary server, the update is fanned out to replica topics on secondary servers.

Why use fan-out?

Having a primary server feed out updates to a number of secondary servers provides a solution where the same topics and data are available from multiple servers. You can use this solution to load balance a large number of client connections across a set of Diffusion™ servers and provide those clients with the same access to data.

How fan-out works

Fan-out is configured on the secondary server or secondary servers in the solution.

Figure 1. Fan-out A fan-out solution comprising one primary server and two secondary servers. The secondary servers have fan-out configured for branches of the topic tree on the primary server.
  • A secondary server connects to a primary server as a client.
  • The secondary server subscribes to a set of topics on the primary server.

    This set of topics is defined by a selector in the configuration of the secondary server.

  • The secondary server replicates the subscribed topics locally.
  • When updates are made to the topics on the primary server, the secondary server receives these updates through the standard pub-sub feature in the same way as any other client of the primary server.
  • The secondary server applies the updates to its replica topics.
  • Any clients subscribed to a replica topic on the secondary server receive the updates through the standard pub-sub feature.
  • If a topic is removed at the primary server, the secondary server removes its replica topic.
  • If a topic is added at the primary server that matches the set of topics subscribed by the secondary server, the secondary server creates a local replica topic.

A secondary server can connect as a client and subscribe to topics on more than one primary server. However, ensure that the secondary server does not attempt to replicate the same topic from multiple sources as this can cause the data on the topic to be incorrect.

Creating topics on the primary server is an asynchronous action, because of this a client or publisher that creates a topic on the primary server receives a completed callback saying that the topic has been created. However, receiving this callback does not indicate that the topic has been replicated by fan-out and created on a secondary server.

Topic aliasing is not supported for topics that are replicated by fan-out. Ensure that aliasing is not enabled at the primary server.

Routing topics and fan-out

To use fan-out with routing topics, the routing subscription handlers for a routing topic must be registered at all secondary servers, but not at the primary server.

Slave topics and fan-out

A primary server will only replicate a slave topic to a fan-out secondary server if the topic is bound to a master topic that also exists on on the primary.

If the master topic is removed from the primary, both the master and the slave will be removed from the secondary.

Topic replication and fan-out

A secondary server cannot replicate the same topic from more than one primary server or multiple times from the same primary server. Validation of the path prefix of the selectors is in place to prevent this occurring, but the use of regular expressions in topic selectors can result in an overlap of replication which can cause problems.

Missing topic notifications generated by subscription or fetch requests to a secondary server are propagated to missing topic handlers registered against the primary servers. For more information, see Using missing topic notifications with fan-out.

Fan-out and load balancers

If you add a secondary server to a load balancer pool before all topics have propagated from the primary server, it can result in a large number of messages being generated, leading to MESSAGE_QUEUE_LIMIT_REACHED errors appearing in the logs.

If you experience this problem, introduce a delay between enabling fan-out and adding any of the secondary servers to a load balancer pool. There is currently no built-in way to determine when propagation is complete, so you will need to experiment to find out how long the delay needs to be for your configuration.

Reconnection and disconnection

You can configure fan-out servers to use the standard reconnect mechanism. If the connection between the secondary server and the primary server is lost, the secondary server can reconnect to the same session. However, if messages are lost between the primary and secondary server, the reconnection is aborted and the session closed. The secondary server must connect again to the primary server with a new session.

If a disconnection between the primary and secondary server results in the session being closed, the secondary server removes all the topics that it has replicated from that primary server. (Only topics explicitly defined by a selector are removed.) Clients subscribing to these topics on the secondary server become unsubscribed. If the secondary server connects again to that primary server with a new session, the secondary server recreates the topics. Clients connecting to the secondary server become resubscribed to the topics.