Package org.igniterealtime.smack.inttest

The Smack Integration Test Framework (sinttest) used to run a set of tests against a real XMPP service. The framework discovers on start-up the available tests by reflection.

Quickstart

You can run the framework against an XMPP service with 

 $ gradle integrationTest -Dsinttest.service=my.xmppservice.org

Note that the service needs to have In-Band Registration (IBR) enabled.

A better alternative to IBR is using XEP-0133: Service Administration to create the throw away accounts used by the integration test framework. Simply use 

 $ gradle integrationTest -Dsinttest.service=my.xmppservice.org \
                          -Dsinttest.adminAccountUsername=admin \
                          -Dsinttest.adminAccountPassword=aeR0Wuub

to run Smack's integration test framework against my.xmppservice.org with an admin account named 'admin' and 'aeR0Wuub' as password.

Configuration

The framework is configured with a standard Java properties file. This file simply contains key/value pairs, which are separated by an equals sign ("="). The most important configuration value is the `service` value, it's also the only required setting.

The file properties can be overridden with Java system properties. The name of a system property that is used by the framework needs to be prefixed with 'sinttest.' (*S*mack *Int*egration *Test* Framework). For example the `service` property becomes `sinttest.service`.

Minimal example 'properties' file

 service = example.org

Another example 'properties' file

 service=example.org
 serviceTlsPin=CERTSHA256:2F:92:C9:4D:30:58:E1:05:21:9A:57:59:5F:6E:25:9A:0F:BF:FF:64:1A:C3:4B:EC:06:7D:4A:6F:0A:D5:21:85
 debugger=console

Framework properties

Properties of the Smack Integration Test Framework
Name Description
service XMPP service to run the tests on
serviceTlsPin TLS Pin (used by java-pinning)
securityMode Either ‘required’ or ‘disabled’
replyTimeout In milliseconds
adminAccountUsername Username of the XEP-0133 Admin account
adminAccountPassword Password of the XEP-0133 Admin account
accountOneUsername Username of the first XMPP account
accountOnePassword Password of the first XMPP account
accountTwoUsername Username of the second XMPP account
accountTwoPassword Password of the second XMPP account
accountThreeUsername Username of the third XMPP account
accountThreePassword Password of the third XMPP account
debugger ‘console’ for console debugger, ‘enhanced’ for the enhanced debugger, or the name of a class that implements SmackDebuggerFactory for a custom debugger
enabledTests List of enabled tests
disabledTests List of disabled tests
enabledSpecifications List of specifications for which to enable tests
disabledSpecifications List of specificatinos for which to disable tests
defaultConnection Nickname of the default connection
enabledConnections List of enabled connection’s nicknames
disabledConnections List of disabled connection’s nicknames
testPackages List of packages with tests
verbose If true set output to verbose
dnsResolver One of ‘minidns’, ‘javax’ or ‘dnsjava’. Defaults to ‘minidns’.
testRunResultProcessors List of class names for generating test run output. Defaults to 'org.igniterealtime.smack.inttest.SmackIntegrationTestFramework$ConsoleTestRunResultProcessor'

Where to place the properties file

The framework will first load the properties file from ~/.config/smack-integration-test/properties.

Running selected tests only

Using enabledTests is is possible to run only selected tests. The tests can be selected on a per class base or by specifying concrete test methods. In the latter case, the methods must be qualified by a (simple) class name.

For example: 

 $ gradle integrationTest -Dsinttest.enabledTests=SoftwareInfoIntegrationTest.test

will only run the test() method of SoftwareInfoIntegrationTest, whereas 

 $ gradle integrationTest -Dsinttest.enabledTests=SoftwareInfoIntegrationTest

would run all tests defined in the SoftwareInfoIntegrationTest class.

Use enabledSpecifications to run all tests that assert implementation of functionality that is described in standards identified by the provided specification-reference.

For example: 

 $ gradle integrationTest -Dsinttest.enabledSpecifications=XEP-0045

would run all tests that are annotated to verify functionality specified in XEP-0045: "Multi-User Chat".

Overview of the components

Package org.igniterealtime.smack.inttest

SmackIntegrationTestFramework

Contains public static void main method, i.e. the entry point for the framework. Here the available integration tests are discovered by means of reflection, the configuration is read and a IntegrationTestEnvironment instance created, which includes the XMPP connections.

AbstractSmackIntegrationTest

The base class that integration tests need to subclass.

AbstractSmackLowLevelIntegrationTest

Allows low level integration test, i.e. every test method will have its own exclusive XMPPTCPConnection instances.

AbstractSmackSpecificLowLevelIntegrationTest

Operates, like AbstractSmackLowLevelIntegrationTest, on its own XMPPConnection instances, but is limited to a particular type of XMPPConnection.

IntegrationTestEnvironment

The environment, e.g. the `XMPPConnections` provided to the integration tests by the framework. Note that for convenience `AbstractSmackIntegrationTest` contains some of those as protected members.

SmackIntegrationTest

An annotation that needs to be added to all methods that represent a single integration test. Annotated integration test methods must not take any arguments, i.e., their parameter count is 0, and should return void as it's not evaluated in any way. The methods are supposed to throw an exception if their integration test fails.

TestNotPossibleException

Can be thrown by test methods or constructors to signal that their test is not possible, e.g. because the service does not support the required feature.

Running the integration tests

Smack's Gradle build system is configured with a special task called `integrationTest`, which means you can run the tests simply with 


 $ gradle integrationTest -Dsinttest.service=my.xmppservice.org

If one of accountOneUsername, accountOnePassword, accountTwoUsername or accountTwoPassword is not configured, then the framework will automatically create the accounts on the service. Of course this requires account registration (IBR) to be enabled. If the accounts got created automatically by the framework, then they will also be deleted at the end of the test.

Implementing Integration Tests

Create a new class which extends AbstractSmackIntegrationTest. Every non-static method, including the constructor, of this class will have two XMPPConnections available to perform the integration tests with: conOne and conTwo. You can use the constructor to check if the XMPP service does provide the required XMPP feature. If it does not, simply throw a TestNotPossibleException.

Test methods must be public, take zero arguments i.e. declare no parameters and be annotated with @SmackIntegrationTest. If the test method is not able to perform a test then it should throw a TestNotPossibleException.

Rules for integration tests

Tests should not leave any traces on the service if they are finished, i.e. the service state at the end of the test must be equal to the state of the beginning. It must be possible to run the tests in parallel.

Why are there two mechanisms to signal that the test is not possible?

Because the XMPP service may provide a component that is required to perform a certain integration test, but that component may not support all features. For example, the XMPP service may provides a PubSub (XEP-0060) component, but this component may not support all features of XEP-0060.

Low-Level Integration Tests

Classes that implement low-level integration tests need to subclass AbstractSmackLowLevelIntegrationTest. The test methods can declare as many parameters as they need to, but every parameter must be of type XMPPTCPConnection. The framework will automatically create, register and login the connections. After the test is finished, the connections will be unregistered with the XMPP service and terminated.

Debugging Integration Tests

A test, like any other code, may not be perfect on the first attempt, and you may require more information in order to ascertain quite what's wrong.

Smack Debugger Options

As described in the package documentation of org.jivesoftware.smack.debugger, there are two built-in debuggers that could surface you more information. Using the 'enhanced' debugger config option listed above, you'll get the Smack Debug Window launching when your tests launch, and you'll get a stanza-by-stanza account of what happened on each connection, hopefully enough to diagnose what went wrong.

Lastly, you can provide a custom debugger, by providing the fully qualified name of a class that implements SmackDebuggerFactory. The provided factory must declare a public constructor that takes no arguments.

Example: 


 $ gradle integrationTest -Dsinttest.service=my.xmppservice.org -Dsinttest.debugger="org.example.MyDebugger$Factory"

Debugging in the IDE

If the output isn't enough, you may need to debug and inspect running code within the IDE. Depending on the IDE, in order to get execution to pause at your breakpoints, you may need to switch your configuration. Instead of running `gradle integrationTest`, instead run the `SmackIntegrationTestFramework` class directly with the same command-line options.

Running Your Own Integration Tests

The framework can be used to run your own tests residing outside of the framework's default package scope. Simply set the testPackages property to a comma separated list of package names where the framework should look for integration tests.

Example: 


 $ gradle integrationTest -Dsinttest.service=my.xmppservice.org -Dsinttest.testPackages=org.mypackage,org.otherpackage

Generating test run reports

By default, the results of the test run is printed to standard-error. You can, however, provide your own processing for the test run results. To do so, create an implementation of SmackIntegrationTestFramework.TestRunResultProcessor and provide its class name to the testRunResultProcessor property. This property takes a comma separated list of class names.

Example: 


 $ gradle integrationTest -Dsinttest.service=my.xmppservice.org -Dsinttest.testRunResultProcessor=org.igniterealtime.smack.inttest.SmackIntegrationTestFramework$ConsoleTestRunResultProcessor