Diffusion 6.6 Release Notes
6.6.0-preview.2 (7 December 2020)
New features in 6.6.0-preview.2
15926: Session properties filter: IN operator
An 'in' operator has been added to session filters to allow for filtering based on an item being present in a list.
24136: Kafka adapter (beta release)
This release includes a beta version of a new adapter for exchange of data between an Apache Kafka cluster and a Diffusion server. The adapter can be used to translate Diffusion topics to Kafka, or Kafka topics to Diffusion, or both at the same time.
Configuration options enable you to rename and restructure the translated data. Monitoring is available via Prometheus.
See the manual for details.
The Diffusion server now accepts connections from clients using the MQTT 5.0 protocol.
MQTT clients can publish and subscribe to Diffusion topics. Both the TCP and WebSocket transports can be used. (Currently Diffusion Cloud only supports the WebSocket transport.)
MQTT sessions are fully integrated with Diffusion security. Authorization rules apply to publication and subscription. Connections can optionally be secured with TLS.
22335: Python client SDK - Basic Features
Diffusion now has a SDK for developing clients in the popular Python language.
In this preview release, the Python SDK supports subscribing to topics, and request-response messaging. More capabilities will be added in the full 6.6 release.
24137: Topic Views : Inserts
The new 'Topic View Inserts' feature allows values from topics other than the source topics being evaluated for a topic view to be inserted into the values of the resulting reference topics. This is enabled by the new 'insert' clause in the topic views DSL. See API documentation for full details.
22702: TimeSeries topics can be updated with a user supplied timestamp
A variation of append has been added to the TimeSeries feature which accepts a timestamp. This allows the user to control the timestamp metadata of the TimeSeries topic event.
Improvements in 6.6.0-preview.2
23858: Value streams deserialise and propagate time series values
Previously events for time series topics were only received by time series event streams added with AddTimeSeriesStream. This has been changed so that all streams will emit the topic value without the related time series metadata, if they are compatible with the event type.
22693: Android jar no longer includes SLF4J API
The Diffusion Android client now no longer provides SLF4J API internally as part of the client. Instead, the client now requires an application dependency on SLF4J API. Specify this in the Gradle file you're using to build your application.
22365: ClientControl can now close other sessions and change client queue conflation of other sessions
Client control feature is now capable of closing session via sessionID or session properties filter, enabling or disabling the client queue conflation via sessionID or session properties filter and set queue event listeners.
22878: Filtered request callback is no longer a stream
PTDiffusionSessionResponseStreamDelegate methods diffusionStream:didFailWithError: and diffusionDidCloseStream: are now deprecated and will be removed in a future release.
24257: APR library update
Following the GLIBC update to v2.32, APR and APR-Utils libraries have been updated to 1.70 and 1.6.1 respectively.
21340: Update streams now respect topic conflation settings and can be used to update topics with DONT_RETAIN_VALUE=true
Update streams now support updating topics with the DONT_RETAIN_VALUE property set to true. Update streams now update these topics by sending whole values to the server instead of deltas.
When creating a new stream, conflation of pending values is now applied based on the conflation property of the topic.
23010: TopicUpdate feature can now be used to update time series topics
Topic update methods can now be used to update time series topics. This allows for creating a topic with an initial value, update constraints and delta streams.
19345: Time series topic query and visualisation
The Diffusion management console now provides an interface to fetch the retained previous values of time series topics. In addition, numeric time series topics are displayed graphically.
23469: Additional sorting and filtering controls for large configuration tables
Several tables in the Diffusion management console which can display many items, for example topic views, now have improved sorting and filtering controls to make locating individual items easier.
Java & Android Client
19053: Removing topics returns the number of topics removed
TopicControl#removeTopics now returns a CompletableFuture bearing a TopicRemovalResult. Calling getRemoveCount returns the number of non-reference topics that were removed as a result of the operation.
23832: Value streams now propagate events from subscribed time series topics with a compatible event type
Previously events for time series topics were only received by time series event streams added with addTimeSeriesStream. This has been changed so that all streams will emit the topic value without the related time series metadata, if they are compatible with the event type.
24481: Datatypes#forClass now returns the binary datatype when called on the Bytes class
In previous releases, Diffusion.datatypes().forClass(Bytes.class) threw IllegalArgumentException.class. From this release, Diffusion.datatypes().forClass(Bytes.class) returns the binary datatype. As a consequence of this change, Bytes instances can now be passed through the messaging API.
23986: Topic creation and removal
The Python SDK now supports topic creation and removal.
23003: The CPU used by a mostly idle server has been reduced
In this release, server multiplexer threads use less CPU when they are not busy. This reduces the CPU load required to run the server when there are few updates.
23936: Topic view scalar path directives now apply to time series JSON topics
The topic view scalar path directive may now be used with time series topics that have a JSON event type as well as normal JSON topics. When used with time series topics the scalar value is always taken from the latest time series event.
Deprecations in 6.6.0-preview.2
22877: The IStream methods of the SendRequestToFilter IFilteredRequestCallback are deprecated
In previous releases, IFilteredRequestCallback.OnError() was called if a SendRequestToFilterAsync operation failed. In addition, the failure was also notified via an exception thrown by the task returned by SendRequestToFilterAsync. IFilteredRequestCallback.OnClose() was never called.
From this release, IFilteredRequestCallback.OnError() is no longer called. Applications should rely on the task returned by SendRequestToFilterAsync to detect errors. Both OnError() and OnClose() methods have been deprecated. In a future release, these methods will be removed from IFilteredRequestCallback and it will be changed not to inherit from IStream.
22879: The onError and onClose methods of the sendRequestToFilter FilteredResponseHandler are deprecated
In previous releases, FilteredRequestCallback.onError() was called if a sendRequestToFilter operation failed. In addition, the failure was also notified via exceptional completion of the sendRequestToFilter result. FilteredRequestCallback.onClose() was never called.
From this release, FilteredRequestCallback.onError() is no longer called. Applications should rely on the Result returned by sendRequestToFilter to detect errors. Both onError() and onClose() methods have been deprecated. In a future release, these methods will be removed from FilteredRequestCallback.
Removals in 6.6.0-preview.2
23392: The Push Notification Bridge has been removed
The Push Notification Bridge has been removed in this release.
The bridge packaged in previous releases can be used against this release. Please contact Push Technology if you are using the bridge, or have requirements to integrate Diffusion with push notification servers.
23114: session_set_maximum_outbound_queue_size removed
Removes session_set_maximum_outbound_queue_size from the API. If you require a different outbound queue size for your session, use diffusion_session_factory_maximum_queue_size
23259: Deprecated Items Removed
Topic details related structures and functions have been removed.
In SVC_NOTIFY_SUBSCRIPTION_REQUEST_T structure, TOPIC_DETAILS (topic_details) has been replaced with TOPIC_SPECIFICATION (topic_specification).
In TOPIC_MESSAGE_T structure, TOPIC_DETAILS_T (details) has been replaced with TOPIC_SPECIFICATION_T (specification).
In SESSION_T structure, _topic_details_refs has been removed.
The file topic_details_types.h has been removed.
The enumeration TOPIC_TYPE_T is now in file topic_specification.h
20517: TopicEventListener/TopicEventStream removed
The deprecated TopicEventListener and TopicEventStream interfaces have been removed from all client SDKs and support for older clients using them has been removed from the server.
23017: TopicUpdateControl feature removed
The deprecated TopicUpdateControl feature has been removed from all client SDKs and support for it in older clients connecting to the server has also been removed.
24258: The ClientServiceConfig compressionThreshold setting has been removed
The ClientServiceConfig compressionThreshold setting has been removed. This setting was no longer used and was deprecated in Diffusion 6.5.
23255: Support for fan-out connections from Diffusion versions before 6.2 removed
The server no longer supports fan-out connections from servers before version 6.2.
20814: Legacy AuthenticationHandler interface removed
The deprecated legacy AuthenticationHandler interface has been removed from all client SDKs and support for older clients using it has been removed from the server.
13044: Server support for SessionDetails services removed
Support for the SessionDetails based API features available via release 5.9 clients has now been removed from the server. Release 5.9 clients will no longer be able to use the getSessionDetails and SessionDetailsListener capabilities with a release 6.6 server.
23116: Legacy AddTopic services removed from Server
A number of addTopic methods which used TopicDetails were removed from all clients in release 6.2. Support for these methods has now been removed which means that pre 6.2 clients will no longer be able to use these methods against a 6.6 server. This also means that clients using EOL releases (specifically 5.9) will no longer be able to add topics at all in a 6.6 server.
23140: getTopicDetails service removed from server
The deprecated getTopicDetails service was removed from all clients in version 6.4. Support for the service has now been removed from the server which means that pre-6.4 clients will no longer be able to use this feature.
19347: Legacy Fetch feature removed
The deprecated legacy fetch mechanism has now been removed from the Topics feature in all client SDKs and from the server (meaning that older clients cannot use it against a 6.6 server). Supporting interfaces (such as Content) have also been removed.
23016: Slave topics removed
The deprecated slave topic type has now been removed from all client SDKs and all support for it has been removed from the server. Older clients will no longer be able to create slave topics. Slave topics stored in persistence files will not be restored.
Fixes in 6.6.0-preview.2
24471: Time series value stream event datatype compatibility problem
Value streams are compatible with parent types. For example, a IValueStream<IJSON> can deserialize values from a String topic type.
IValueStream<IEvent<IJSON>> did not propagate the values for time series topics with the String event type. This has now been fixed.
24305: Selectors with descendant qualifiers not evaluated
Addresses an issue where selector strings, containing descendant qualifiers, were not evaluated in the C client. This has now been resolved.
23817: Can't type negative values in console topic editor
The topic value editor in the Diffusion management console did not accept negative values for int64 and double topics. This has been fixed.
23865: Sessions tabs defaults to metrics, even if not permissioned
The Diffusion management console presented an error message to users who browsed to the Sessions tab without the required permissions to view the default (session metrics) tab. The console will now show the sessions browser in this case.
23968: Disallow creating time series topics with DONT_RETAIN_VALUE
The Diffusion management console allowed users to configure new time series topics with the DONT_RETAIN_VALUE topic property, which is not allowed for time series topics and would be ignored. This property is no longer listed for time series topics.
24505: Open sessions tab shows wrong session as having been closed
In the Diffusion management console, after using the open sessions browser to close a session, the table entry for a different session would be removed. The correct entry is now removed in all cases.
Java & Android Client
23956: Flow control can stall update streams under high load
In previous releases, a bug could cause a Java client using update streams to never recover from flow control. This caused updates to stall. The bug has been fixed in this release.
24399: Time series value stream event datatype compatibility problem
Value streams are compatible with parent types. For example, a ValueStream<JSON> can deserialize values from a String topic type.
ValueStream<Event<JSON>> did not propagate the values for time series topics with the String event type. This has now been fixed.
23385: Client should cascade to XHR protocol by default
23895: Documented incorrect ordering of arguments to PartialJSON.with()
The order of arguments of PartialJSON.with() is now documented correctly
23896: Cannot use numbers/int64/float in JSON constraints
Previously int64 data was serialized inconsistently and for 32 bit integers and wrongly for negative 64 bit integers. This caused partial JSON constraints to fail for integer values. It also prevented negative 64 bit integers from being passed between clients. Integers are now serialized correctly and in line with other clients. Partial JSON constraints can be used with the Int64 datatype.
24095: Error when client attempts to establish an insecure connection from a secure (https) origin
Error "Failed to connect TypeError: Cannot read property 'close' of undefined" could occur when attempting to establish an insecure connection from a secure (https) origin. This problem has now been resolved.
24510: Buffer implementation not supported outside of NodeJS
Previously, there was no access to the Buffer API when running the client in a browser context. The Buffer API is now available through diffusion.buffer.
24035: Failures to process locked security resources
The API allows security roles and system authentication principals to be locked by a particular principal. Due to bugs in previous releases, processing of locked resources could fail:
A server could not boot against a Security.store or SystemAuthentication.store configuration file containing one or more lines that modify a locked resource. The work around was to remove the offending lines.
In a cluster, a modification to a locked resource was not properly replicated. Instead, an error message would be logged by all servers receiving the replicated update.
Both bugs have been fixed in this release. Locked security roles and system authentication principals now function as designed.
24032: Topic view publishes new value to old topic before removing it if scalar mapping is used in a simple (non-derived) view mapping
A topic view specification that contained a scalar path mapping but no other JSON type value mappings could result in a new value on the source topic being published to the reference topic before the removal of the reference topic due to a change to the indicated scalar value.
This problem has now been resolved.
24092: Incorrect handling of regular expression quotes in topic selector expressions
Split-path pattern and full-path pattern topic selectors allow the use of regular expression quotes (\Q, \E). In previous releases, the quotes were handled incorrectly: any backslash (\) character between the quotes was interpreted as the end of the literal section.
This bug has been fixed in this release. Now everything between \Q and \E is correctly treated as literal characters.
24231: Potential failure to process time series updates
In previous releases, a rapid sequence of updates to time series topics could fail to be processed correctly, leading to data corruption. This bug has been fixed in this release.
24234: Automatic topic removal policy using "when this session closes" - topic can be removed while session still exists
If a topic had the 'when this session closes' REMOVAL clause and the creating session then failed over to another server in the cluster, the topic would be removed. This is incorrect: the topic should remain as long as the creating session does.
A similar problem could also occur if the creating session had its session properties changed in any way.
These problems have now been resolved.
24566: Fallback streams do not support propagation of time series values
Release 6.6 Preview 1 introduced support for receiving values from time series topics with compatible ValueStreams from other topic types. This support has been extended to the dotnet client in Preview 2. Fallback stream support for this feature will be added in a future release.