Wednesday, March 29, 2017

Future<IQ> - About the New Asynchronous API in Babbler

This is a topic which is long overdue, but still pretty interesting: The new asynchronous, non-blocking API for IQ requests in Babbler (since version 0.7).

The Problem

Until the previous release, all IQ-based API was synchronous and did block until the IQ response has been received.

Lets have a look at what that means and how Last Activity (XEP-0012) of an entity was retrieved to illustrate the problem:

LastActivityManager lastActivityManager = xmppSession.getManager(LastActivityManager.class);
LastActivity lastActivity = lastActivityManager.getLastActivity(Jid.of("juliet@example.com/balcony"));

The method getLastActivity() sends out the IQ request and then waited (blocked) a few milliseconds or even seconds for the response or until a timeout happened.

Having blocking operations is of course resource consuming because you have to dedicate a thread for it, which however is blocking most of the time while waiting on the operation to finish. It's the same issue as with blocking IO and the reason why NIO exists.

Doing multiple blocking IQ queries in parallel means, you have to create a thread for each query.

If you want to do such an IQ query from a JavaFX application you also had to write a lot of boilerplate code like this:

Task<LastActivity> task = new Task<LastActivity>() {
    @Override
    protected LastActivity call() throws Exception {
        LastActivityManager lastActivityManager = xmppSession.getManager(LastActivityManager.class);
        return lastActivityManager.getLastActivity(Jid.of("juliet@example.com/balcony"));
    }
};
task.stateProperty().addListener((observableValue, state, state1) -> {
    switch (state1) {
        case SUCCEEDED:
            updateUI(task.getValue());
            break;
        case FAILED:
            task.getException().printStacktrace();
            break;
        default:
            break;
    }
});
new Thread(task).start();

You don't want to block the UI thread and therefore need to run blocking operations in a background task.

Furthermore the blocking API in Babbler was not interruptible because it didn't throw InterruptedException. Of course we could have solved the interruptible issue easily, but you still would have the drawbacks of a blocking API.

Futures to the Rescue

Instead of waiting for the response and then returning the result, all IQ-based APIs now return a java.util.concurrent.Future:

Future<LastActivity> lastActivityFuture = lastActivityManager.getLastActivity(Jid.of("juliet@example.com/balcony"));

The method call is now asynchronous, it no longer blocks and passes control immediately back to the caller!

As with every Future, you can get the result with its get() method:

LastActivity lastActivity = lastActivityManager.getLastActivity(jid).get();

or with a timeout:

LastActivity lastActivity = lastActivityManager.getLastActivity(jid).get(5, TimeUnit.SECONDS);

You might ask, what we've gained now, because the get() method is blocking again and usually you need the IQ result anyway.

Well, that's true. One part of the answer is that we gain interruptibility and the other part is that the returned result is not only a Future, but also a java.util.concurrent.CompletionStage (Java 8's new toy).

CompletionStage<LastActivity> lastActivityFuture = lastActivityManager.getLastActivity(Jid.of("juliet@example.com/balcony"));

It basically allows you to react asynchronously when the result is present, i.e. when the Future is done. Some frameworks like Guava already have such a concept of a "Listenable Future", now it's part of the JDK.

Taking our JavaFX example from above, updating the UI with the result as in the above example can now become a simple one-liner:

lastActivityManager.getLastActivity(jid).thenAcceptAsync(this::updateUI, Platform::runLater);

It sends the IQ request and later when the response is received it asynchronously executes the updateUI method in the JavaFX thread.

No more blocking, no more extra threads, everything is asynchronous!

Even better:

CompletionStages can be chained together. There are use cases, which require multiple IQ queries like Service Discovery or File Transfer. They can then be composed together into one:

CompletionStage<Boolean> isSupported = xmppClient.isSupported(LastActivity.NAMESPACE, jid);
CompletionStage<LastActivity> lastActivityFuture = isSupported.thenCompose(result -> {
    if (result) {
        return lastActivityManager.getLastActivity(jid);
    } else {
        throw new RuntimeException("XEP-0012 not supported by" + jid);
    }
});

This code first checks if Last Activity is supported (using Service Discovery) and only if it is, queries the entity.

File Transfer is pretty complicated, with a lot of queries going on. This pseudo-code example illustrates the power of composing asynchronous calls (IBB fallback not shown here):

CompletionStage<ByteStreamSession> future =
initiateStream() // Initiate a file transfer stream with somebody
    .thenCompose(streamInitiation -> discoverStreamHosts() // When accepted, discover SOCKS5 stream hosts
        .thenCompose(streamHosts -> letReceiverChoseOne() // Query the receiver and let him choose a stream host
            .thenCompose(streamHostUsed -> activateStreamHost() // When receiver responds with the chosen stream host, activate it 
                .thenApply(result -> createByteStreamSession())))); // After activation, create a stream session.

For convenience there's also a class AsyncResult, which implements both interfaces.

I think asynchronous programming is the future :-) and this is a first, but huge step in the right direction.

Thursday, March 16, 2017

Babbler Version 0.7.4 released

Version 0.7.4 of the Java XMPP library has been released to Maven Central!

It turned out there was a rare deadlock, when using Stream Management. It happened only rarely and was hard to spot, but when it did, it was of course a blocker.

Here's the full changelog:

  • Resolve rare deadlock when using Stream Management
  • Rework how WebSocket connections are closed
  • Don’t let a stream error close the stream immediately, but instead wait for the closing stream element and then close the connection.
  • Increase performance of IBB stream
  • Prevent rare, but possible NullPointerException after sending stanzas.
  • Fix error when using pages (XEP-0141) within data forms (XEP-0004)
  • Reset nick to null, if entering a chat room fails

Saturday, February 11, 2017

Babbler Version 0.7.3 released

I've released version 0.7.3 of the Java XMPP library. This is primarily a "bug fix and improvements" release and is compatible with previous 0.7.x releases. Here's the changelog:
  • Use single equals sign (“=”) for zero-length data in SASL, as per RFC 6120 § 6.4.2
  • Allow configuring a custom stream host and skip proxy discovery then for SI file transfer.
  • Implement WebSocket pings/pongs.
  • Fix WebSocket’s proxy URI construction.
  • Use connect timeout for WebSocket connections.
  • XEP-0198: Send an ack right before gracefully closing the stream (i.e. update to version 1.5.2).
  • MUC Room “enter” events should fire for oneself entering the room as well.
  • Use java.text.Collator for String-based default comparison.
  • XEP-0066: Use URI instead of URL.
  • Fix XMPP Ping in External Components, which broke the connection.
  • Jid.asBareJid returns this if it is already bare, reducing GC pressure.
  • connect() method should not throw CancellationException
  • Check if the connection has been secured (if configured) before starting to authenticate.

Maven coordinates

<dependency>
    <groupId>rocks.xmpp</groupId>
    <artifactId>xmpp-core-client</artifactId>
    <version>0.7.3</version>
</dependency>
<dependency>
    <groupId>rocks.xmpp</groupId>
    <artifactId>xmpp-extensions-client</artifactId>
    <version>0.7.3</version>
</dependency>

Friday, September 9, 2016

Version 0.7.2 Released

A new bugfix version has just been released to Maven Central with the following issues resolved:
  • Fix reconnection issue, when using multiple connection methods per session.
  • Improve and fix stanza acknowledging and Stream Management
    • Add Delayed Delivery (XEP-0203) extension to stanzas, which are resent automatically later (when reconnected again)
    • Always resent all unacknowledged stanzas after login, not only after stream resumption.
    • Highlight StreamManagement’s request / answer pairs in VisualDebugger.
    • Update XEP-0198 Stream Management to version 1.5 (respect the ‘h’ attribute in the failed element)
  • Wait for the roster response before sending initial presence during login, to prevent receiving presence information from yet unknown contacts.
  • Make sure asynchronous method calls do not block (affected only few methods for avatars and entity capabilities)
  • Use the hostname instead of the domain for SASL clients (i.e. use the Sasl.createSaslClient API correctly as per the documentation, may affect DIGEST-MD5 authentication).
  • Call SaslClient.dispose() when SASL authentication has completed.
  • Include the requesting IQ in NoResponseException, when doing IQ queries.
  • XEP-0184: Add the sender of a receipt to the MessageDeliveryEvent.
  • Allow event consumption for outbound stanzas, which prevents the stanza to be sent.
  • Make stream feature negotiation more stable.
  • Minor graphical fixes in VisualDebugger.
  • Add API to include the hash and mime type in File Transfer offers.
  • Add API to create a chat session with a thread id.
  • Immediately complete (IQ-)queries if sending failed and don’t wait on the timeout.

Thursday, August 11, 2016

Babbler 0.7.1 (Bugfix) Released

A bugfix version of the XMPP client library has been released to Maven Central: 0.7.1

Here's the changelog:

  • Discovering services should not fail immediately if one sub-query fails.
  • Make sure abnormal WebSocket disconnections trigger the reconnection.
  • Make sure RECONNECTION_SUCCEEDED event is triggered for external components.
  • Add listeners to listen for successful or failed send operations.
  • Add public constructor for the SASL challenge class.
  • Add public constructors to SASL Failure class.
  • Make sure to not write XMLConstants.XML_NS_URI to XML elements (FasterXML Aalto’s XMLStreamWriter implementation writes it)
  • Add DataForm.Field#getValue() and implement toString() method.
  • Add convenient API to compare two MUC affiliations and roles (i.e. Affiliation.OWNER.isHigherThan(Affiliation.ADMIN))
  • Compare presences of MUC occupants in the Occupant’s Comparable implementation.
  • Minor performance improvement by using a ListIterator in collection based result sets.
  • Discover PubSub services by identity, not by feature name (it’s more reliable)
  • Add nextPage() and previousPage() method and refine the naming of other methods in result set management (e.g. having forCount() and forItemCount() was confusing)

Sunday, June 5, 2016

Babbler Version 0.7.0 Released

Version 0.7.0 of the XMPP Java library has just been released!

As always you can find the artifacts in Maven Central:

<dependency>
    <groupId>rocks.xmpp</groupId>
    <artifactId>xmpp-core-client</artifactId>
    <version>0.7.0</version>
</dependency>
<dependency>
    <groupId>rocks.xmpp</groupId>
    <artifactId>xmpp-extensions-client</artifactId>
    <version>0.7.0</version>
</dependency>

Release highlights are support for WebSocket connection, the new XMPP Address Format (RFC 7395), asynchronous API for IQ queries and XEP-0198: Stream Management (which is still a bit experimental).

Some API has also been revised for less ambigious and easier usage, e.g. timeouts are now represented by java.time.Duration instead of a int/long (which somtimes were seconds, sometimes milliseconds) and the xml:lang attribute is now represented by java.util.Locale instead of String to prevent improper usage.

Here's a more complete list of changes:
  • Add support for XEP-0198: Stream Management
  • Add support for WebSocket connection method (RFC 7395).
  • Update Jid class to the new XMPP Address Format (RFC 7622)
  • IQ queries can now be executed asynchronously (non-blocking) using Java 8’s java.util.concurrent.CompletableFuture API.
  • Represent xml:lang attributes as java.util.Locale, not as String.
  • Represent timeouts as java.time.Duration instead of int/long for better clearness.
  • Add a very minimalistic DNS resolver for resolving SRV and TXT records in order to remove the dependency to com.sun.* classes.
  • Add more ReconnectionStrategy implementations.
  • Check connected state of socket before connecting (to prevent SocketException when a SocketFactory provides a connected socket)
  • Add XmppSession#isAuthenticated() method.
  • Add static XmppSession#addCreationListener() method to allow to listen for newly created sessions.
  • Update XEP-0080 to version 1.9 (add altaccuracy element).
  • Add API to destroy a MUC room without a reason.
  • More documentation, e.g. clarify the use of ConnectionConfiguration#secure()
  • Don’t include an empty body in Message Delivery Receipts.
  • Add correct XML names to component namespace stanzas.
  • Eagerly release unused port to prevent ports-leaks due to delayed GC
  • Improve thread-safety during connect() and login()
  • Fix encoding issues, caused by missing UTF-8 encoding, mainly in the debugger.
  • XEP-0033: Address should have extensions.
  • Add workaround for a JDK bug causing memory issues and high CPU.
  • Add documentation for custom SASL authentication.

Sunday, January 3, 2016

Precis (RFC 7564/7613) - A Java Implementation

Since a few months, there are a few new specifications around concerning internationalized strings in application protocols, which obsoletes the old Stringprep specification (RFC 3454).
  • RFC 7564: PRECIS Framework: Preparation, Enforcement, and Comparison of Internationalized Strings in Application Protocols
  • RFC 7613: Preparation, Enforcement, and Comparison of Internationalized Strings Representing Usernames and Passwords
  • RFC 7700: Preparation, Enforcement, and Comparison of Internationalized Strings Representing Nicknames

These are also used by XMPP, especially by the new Address Format specification, RFC 7622 (The local and resource part must conform to Precis profiles).

After working on a Java implementation for Precis for some time, I'd finally like to announce the first release of the open source software, which supports all of the three Precis specifications:

Project site with more information

Maven coordinates:

<dependency>
    <groupId>rocks.xmpp</groupId>
    <artifactId>precis</artifactId>
    <version>0.1.0</version>
</dependency>

Anybody who likes to upgrade the obsolete Libidn's Stringprep class is invited to give it a try.