Integration Tests With Arquillian Extensions on the Payara Platform

Arquillian is one of the most classical integration test frameworks for JVM (Java Virtual Machine)-based applications, currently maintained by Red Hat. Its main particularity consists in the fact that, in order to facilitate integration testing through the maven lifecycle, it allows the deployment of the tested applications on an embedded, managed, or remote Jakarta EE-compliant container, like Payara.

From Manual To Automated Integration Testing

Historically speaking, the integration testing process used to be a highly manual activity, consisting of deploying the application on the application server before running the test cases. This is especially due to the utilization of building tools like maven which default lifecycle provisions the integration test phase, named verify, just before install and deploy. This means that in-container integration tests cannot be performed in an out-of-the-box manner with such tools, because they are run before the deployment phase, despite the fact that, in order to be run, they require applications to be deployed. So, we find ourselves in a kind of circular problem here: in order to test the application we need to deploy it first, and, in order to deploy it, we need to test it first. This is why integration tests were traditionally performed manually.

Things changed with the introduction of automated testing based on the JUnit framework. This framework defines the notion of the unit test as a piece of code meant to prove that a given specific functionality in the application works as expected. And since testing such functionalities in complete isolation wouldn't be possible, due to their dependence on other software components or on complex infrastructure elements, a new technique, called mocking, has emerged. 

This technique consists of using mock objects that act as mediators between the testing code and the tested components. In fact, instead of calling real components, the test cases call now the mock objects that merely assert that the correct methods are invoked, with the correct signatures, returning the correct results. This way we, as developers, are able to get rid of complex dependencies of the tested code and to test it in isolation, proving this way that it should work, as far as it runs in a fully deterministic context, where all the required constraints are satisfied.

The problem with this approach is that it not only forces us to refactor the code such that it complies with the mock object's interfaces, but additionally, it doesn't allow us to test real objects; rather, mock ones. As a matter of fact, proving that the code is working as expected, as soon as the execution context is fully deterministic and all the constraints are satisfied, is not very useful because we are seldomly in such a context. Aside from that, the ultimate goal of the testing activity is to prove that, in fact, our software components are flexible enough and work as expected, whatever the different variations, caused by a possible unstable context, might be.

Finally, there is no consistent way to automatically unit test complex Jakarta EE software components like Servlets, EJBs, JPA, and others, because:

• Testing them in-container requires deployment, and this isn't possible as the automatic tools test phase runs before the deployment.
• Testing them in isolation requires us to mock lots of dependencies and, by doing that, we aren't testing our real components; just mock objects.

Introduction to Arquillian

As opposed to other testing frameworks, Arquillian does all the required plumbing of container management and deployment, by providing the following elements:

• A test runner platform, such as JUnit 4, JUnit 5, Jupiter, or TestNG
• A set of containers such as Payara, Wildfly, Glassfish, Tomcat, WebLogic, WebSphere, etc.
• A set of test enrichers aiming at leveraging the test cases integration into the above containers
• The ShrinkWrap library facilitates the deployment descriptors' definition and loading. Notice that ShrinkWrap is an external library and, as such, it doesn't belong to Arquillian, even if it is typically used in Arquillian-based integration tests.

An Arquillian test case is divided into two main areas:

1. The deployment section, which packages and deploys the tested application on the application server
2. The test section, which performs the real testing operations based on the selected runner platform (JUnit or TestNG).

As for the supported application server containers, they come in the following flavors:

• Embedded containers running in the same JVM as the test cases
• Managed containers running in another JVM; then the test cases may be started and stopped by them
• Remote containers that run in a completely independent manner

Arquillian Cube

We have described above a typical scenario of using the Arquillian Core framework in integration tests. The advent of Linux container-based technologies, like Docker and OCI (Open Container Initiative), have demonstrated that these platforms make perfect testing environments, thanks to their reliable and reproducible attributes. But using Linux containers with Arquillian integration tests (i.e., Docker containers) requires developers to deal with hosts machines and IP addresses, as well as to adapt the building scripts such that to provide specialized steps that create images or containers and fire them up (i.e., by manually running docker or docker-compose commands). And even if running manually such commands might be a good approach, we need to remember that integration tests should be self-executing as much as possible and not require manual intervention or complicated run times.

Enter Arquillian Cube: This is an Arquillian extension that helps manage Docker containers and images in Arquillian-based integration tests. It uses a similar approach to the one of Arquillian Core for application servers but adapted for Docker containers. It may be used in cases as follows:

• Preparing testing environments
• Testing Dockerfile build operations
• Validating Docker Compose compositions
• Performing white and black box testing

The savvy reader has certainly noticed some similarities with another testing tool extensively used currently with Docker containers: testcontainers. Those readers would probably be pleased to know that Arquillian Cube could be considered as a kind of combination of Arquillian Core and testcontainers. The figure below shows the Arquillian Cube lifecycle.

As shown by the figure above, before executing integration tests, Arquillian Cube reads a Docker Compose file and starts all the Docker containers in the right order. The extension waits then until all the services are up and running so they're able to receive incoming connections. Then, the tests are executed, and, once finished, all the running Docker containers are stopped and removed.

The Proof of Concept

In order to demonstrate how the integration testing process could be conducted with Payara application servers, we provide a proof of concept (POC). The runner platform that we're using in this POC is JUnit 4 and 5 (using a TestNG runner platform is very similar). Hence, our project is structured around two maven modules named with-junit-4 and, respectively, with-junit-5.

In order to experience this POC you need, of course, a Docker daemon running on your box. Under the hood, this daemon uses Linux sockets to communicate with clients. This is perfect for Linux users. However, if you use Windows or Mac, then boot2docker or docker-machine might also be required.

Arquillian Cube supports three flavors of integration tests:

1. The deployable integration tests are the ones with which the Arquillian Core users are already used. They require the deployment section that we mentioned previously where, thanks to the ShrinkWrap library, a minimal WAR or JAR, containing the components to be tested, is created and deployed on the target application server. The only difference here is that the target application server is running, in the case of Arquillian Cube, in a separate Docker container.
2. The container object-based integration tests: Container object is a pattern coined by the Arquillian community which consists in encapsulating, in an object-oriented manner, data and actions related to containers, such that they might be reused in different integration tests.
3. The DSL (Domain Specific Language)-based integration tests: Arquillian Cube provides a DSL that can be used to dynamically create and instantiate containers.

Our POC includes a Maven sub-project illustrating each one of the flavors above available with both supported runner platforms: JUnit 4 and 5. Let's have a look at these integration tests.

The Deployable Integration Tests

In order to implement our integration tests, we consider a very simple use case, the one of a "Hello, world!" servlet whose code is shown below:

@WebServlet("/hello")
public class HelloServlet extends HttpServlet
{
  @Override
  protected void doGet(HttpServletRequest req, HttpServletResponse resp)
    throws ServletException, IOException
  {
    PrintWriter writer = resp.getWriter();
    writer.println("Hello World");
  }
}

This same code will be reused across all the POC projects. In order to test it, we implement the following integration test:

@RunWith(ArquillianConditionalRunner.class)
public class PayaraServerFullIT
{
  @ArquillianResource
  private URL url;

  @Deployment(testable = false)
  public static WebArchive onPayara()
  {
    return ShrinkWrap.create(WebArchive.class, "hello.war").addClass(
      HelloServlet.class);
  }

  @Test
  public void testOnPayaraShouldReturnOk() throws MalformedURLException
  {
    given()
      .contentType(ContentType.JSON)
      .get(new URL(url + "/hello"))
      .then()
      .assertThat().statusCode(200)
      .and()
      .body(containsString("Hello World"));
  }
}

This is a deployable integration test having the already-discussed two sections: the deployment and the test one. Here, the Arquillian users will recognize the @Deployment annotation which leverages the ShrinkWrap library to create a minimalistic WAR containing the servlet class to be tested. Once created, this WAR will be deployed, as we'll see in a moment.

The @ArquillianResource annotation above is called an enricher, and it aims at injecting some new powerful operations that are very useful when it comes to communicating with the Docker daemon. In our case, this annotation is used in order to enrich the integration test with a Docker client reference which allows the developer to extract the servlet URL. Since containers have these characteristics of being ephemeral, the developer cannot anticipate at the build time such things as IP addresses or TCP ports, as these details will be only known at runtime; hence, the use of the enrichers.   

Last but not least, our integration test uses the well-known REST-assured library to test the servlet. Let's have a look now at the plumbing in arquillian.xml that describes the Docker images and containers to be used as well as their associated details.

<?xml version="1.0"?>
<arquillian xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xmlns="http://jboss.org/schema/arquillian"
            xsi:schemaLocation="http://jboss.org/schema/arquillian
    http://jboss.org/schema/arquillian/arquillian_1_0.xsd">
  <extension qualifier="docker">
    <property name="serverVersion">1.41</property>
    <property name="definitionFormat">CUBE</property>
    <property name="dockerContainers">
      payara:
        image: payara/server-full:5.2022.4-jdk11
        exposedPorts: [4848/tcp, 8080/tcp]
        await:
          strategy: polling
        env: []
        portBindings: [4848/tcp, 8080/tcp]
    </property>
  </extension>
  <container qualifier="payara">
    <configuration>
      <property name="adminPort">4848</property>
      <property name="adminUser">admin</property>
      <property name="adminPassword">admin</property>
      <property name="adminHttps">true</property>
      <property name="authorisation">true</property>
    </configuration>
  </container>
</arquillian>

The container XML element in the listing above is already known by the developers familiar with Arquillian Core. It defines the Arquillian adapter which connects to the application server used to deploy the tested components. Here we're using an application server whose qualifier is "payara", listening on the TCP port 4848, having the credentials "admin/admin" and leveraging HTTPS as a communication protocol. 

In this example, we're using the Payara server. This requires some explanations. The Payara server has an operating mode known as "secure admin". When this operating mode is enabled, the communication between the CLI (Command Line Interface) and the DAS (Domain Administration Server) is encrypted through SSL/HTTPS. When using a vanilla Payara server, as it may be downloaded by anyone from the company's website, the "secure admin" operating mode is disabled by default. However, the Docker images provided by the company are all "secure admin" enabled. This is because in a remote deployment model where the DAS and the CLI might be running on different nodes, as this is the case of any Docker containers-based deployment, the only way to communicate between the client and the server is HTTPS; hence, the property adminHttps in our configuration file above.

The section labeled extension is new and specific to Arquillian Cube. The qualifier docker is a constant and it is mandatory. The serverVersion element defines the version of the API implemented by the Docker daemon running on the executing machine.

Arquillian Cube supports two kinds of notations:

• A Docker Compose-compliant notation
• A Docker Cube-specific notation

While the Docker Compose notation is ubiquitous nowadays and known unanimously, the specific Cube notation is more powerful as it allows to express constructs that cannot be defined in the Docker Compose notation. Here, we're using the Cube notation, as defined by the property definitionFormat.

Finally, the XML element dockerContainers defines the Docker configuration. We're using the image available on DockerHub under the tag payara/server-full:5.2022.4-jdk11. At the time of this writing, Payara Server 6 was still in Alpha release; hence, we're using Payara 5 here. The exposedPorts and portBindings elements have the same meaning as in their counterpart Docker Compose notation. Special attention should be accorded to the property await which allows the definition of a waiting strategy. Starting an application server is an operation that might take some time and the Payara server is no different. Hence, the integration tests should wait until the container has fully started before trying to connect to it. This can be modeled through the notion of await startegy. There are several supported strategies, as explained by the Arquillian Cube documentation. Here we're using the polling one, consisting in testing one by one all the exposed TCP ports until they are in a listening status. This is done with the Linux ss command or, if this command isn't installed on the running Docker image, with ping.

In order to run our test case, we need to proceed as follows:

1. Export the self-signed X.509 certificate from the image's trust store and import it into one of our current JVM. As explained above, the Payara Docker image comes with the "admin server" option enabled. Hence, in order to deploy applications to the application server running in the container, we need HTTPS. The image contains a self-signed certificate in its trust store that we need to import into our JVM trust store such that we trust its genuineness. This might be done using the following script:
   
   Shell

    

   JAVA_HOME=$(dirname $(dirname $(readlink -f /etc/alternatives/java)))
   docker exec -ti payara5 /bin/bash -c "keytool -storepass changeit -export -alias s1as -keystore ./appserver/glassfish/domains/domain1/config/cacerts.jks -rfc -file s1as.cer"
   docker cp payara5:/opt/payara/s1as.cer .

Here we define the JAVA_HOME environment variable and, after that, we run the keytool command on the guest operating system, such as to export the self-signed certificate from the server trust store. Once exported, the self-signed certificate is copied onto the host machine.

2. Build the project using the following command:

mvn -DskipTests -DskipCertImport=false clean package

Here we skip the execution of the unit tests and, by using the property certSkipImport, we import the self-signed certificate into our JVM trust store. This is done thanks to the keytool-maven-plugin, as shown below:

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>keytool-maven-plugin</artifactId>
  <executions>
    <execution>
      <id>import-certificate</id>
      <goals>
        <goal>importCertificate</goal>
      </goals>
      <phase>package</phase>
      <configuration>
        <skip>${cert.skip.import}</skip>
        <file>${basedir}/src/test/resources/s1as.cer</file>
      </configuration>
    </execution>
    <execution>
      <id>delete-alias</id>
      <goals>
        <goal>deleteAlias</goal>
      </goals>
      <phase>clean</phase>
      <configuration>
        <skip>${cert.skip.delete}</skip>
      </configuration>
    </execution>
  </executions>
  <configuration>
    <keystore>${env.JAVA_HOME}/lib/security/cacerts</keystore>
    <storepass>changeit</storepass>
    <alias>s1as</alias>
    <keypass>changeit</keypass>
    <noprompt>true</noprompt>
  </configuration>
</plugin>

3. Now, once the self-signed certificate has been imported, you can run the integration tests:

mvn verify

This should work as expected. If you experience PKIX exceptions, then it means that you didn't correctly import the self-signed certificate.

Notice the differences between the two runner platforms JUnit 4 and JUnit 5:

@RunWith(ArquillianConditionalRunner.class) // JUnit 4
...
@ExtendWith(ArquillianExtension.class)  // JUnit 5  

The Container Object-Based Integration Tests

This method of writing integration tests is based on the definition of the so-called "container objects". These are Java objects encapsulating specific properties and methods relative to the definition of the Docker containers used by the integration tests. Here is the listing of such a "container object:"

@Cube(value = "payara5", portBinding = {"4848->4848/tcp", "8080->8080/tcp"})
@CubeDockerFile
public class PayaraContainer
{
  @HostIp
  private String host;
  @HostPort(8080)
  private int port;

  public int getPort()
  {
    return port;
  }

  public String getHost()
  {
    return host;
  }
}

This code is leveraging the JUnit 5 platform runner. An equivalent example with JUnit 4 is provided as well.

Here we're defining a "container object" flagged with the annotation @CubeDockerFile. This annotation flags a Java class as a "container object." The @Cube annotation defines a Docker container named payara5 and using the mentioned portBinding. Then, using enrichers like @Host and @Port, we encapsulate in this class all the required details relative to our Docker container.

The mentioned container is initiated based on the Docker image created via the following Dockerfile:

FROM payara/server-full:5.2022.4-jdk11
COPY container-object-junit-5.war /opt/payara/deployments

Here we extend the same payara/server-full:5.2022.4-jdk11 image and we add to it the war to be deployed. By doing that, we take advantage of the auto-deployment feature of the Payara server, thanks to which any archive copied in the /opt/payara/deployments directory is automatically deployed.

Once this "container object" is defined, we can use it in our integration tests as follows:

@ExtendWith(ArquillianExtension.class)
public class PayaraContainerIT
{
  private static final String APP_URL = "http://%s:%d/container-object-junit-5/hello";
  private URI uri;

  @Cube
  private PayaraContainer payara;

  @BeforeEach
  public void before() throws MalformedURLException
  {
    uri = UriBuilder.fromUri(
        String.format(APP_URL, payara.getHost(), payara.getPort())).build();
  }

  @AfterEach
  public void after()
  {
    uri = null;
  }

  @Test
  public void testOnPayaraShouldReturnOk()
    throws MalformedURLException, InterruptedException
  {
    sleep(5000);
    given()
      .contentType(ContentType.JSON)
      .get(uri)
      .then()
      .assertThat().statusCode(200)
      .and()
      .body(containsString("Hello World"));
  }
}

The code above shows how a "container object" may be injected exactly like any other Java-managed bean, via the @Cube annotation. Notice that this annotation has two meanings, depending on the context where it is used:

• A cube definition, if it defines properties like value, portBinding, etc.
• A cube reference when used without parameters; this reference is injected into the client code

Again, the differences between the JUnit 4 and the JUnit 5 platform runners are minimal, as shown previously.

The DSL Integration Tests

As opposed to the "deployable" and the "container object" Arquillian Cube integration tests, the DSL ones require neither an XML configuration file nor a dedicated Java class to encapsulate the details of the container. Instead, they benefit from a DSL created by the Arquillian community which allows the developer to generate Cube instances using a generic container. Here is how:

@RunWith(ArquillianConditionalRunner.class)
public class PayaraServerFullIT
{
  private static final String APP_URL = "http://%s:%d/container-object-dsl-junit-4/hello";
  private URI uri;

  @Before
  public void before() throws MalformedURLException
  {
    uri = UriBuilder.fromUri(
        String.format(APP_URL, payara.getIpAddress(),
          payara.getBindPort(8080)))
      .build();
  }

  @After
  public void after()
  {
    uri = null;
  }

  @DockerContainer
  Container payara = Container.withContainerName("payara5")
    .fromBuildDirectory(System.getProperty("user.dir"))
    .withPortBinding(8080)
    .withPortBinding(4848)
    .withVolume(System.getProperty("user.dir") + "/target/container-object-dsl-junit-4.war",
      "/opt/payara/deployments/container-object-dsl-junit-4.war")
    .build();

  @Test
  public void testOnPayaraShouldReturnOk()
    throws MalformedURLException, InterruptedException
  {
    sleep(5000);
    given()
      .contentType(ContentType.JSON)
      .get(uri)
      .then()
      .assertThat().statusCode(200)
      .and()
      .body(containsString("Hello World"));
  }
}

As you can see, this code looks very similar to the one by testcontainers, in the sense that using the @DockerContainer annotation allows you to directly define the required Docker images and containers without any configuration files and without any help from Docker itself or Docker Compose. Accordingly, this code creates a Docker container named payara5 by building the image defined by the Dockerfile in the current working directory, by binding the mentioned TCP ports, and by mounting the WAR file to be tested to the server's auto-deployment directory, as we already did in the previous example.

This method of writing integration tests is very handy; however, it doesn't offer the same rich functionalities as the Cube notation used with the deployable case. As a matter of fact, we cannot define here awaiting strategies as we did previously; hence, the statement sleep(5000) in the REST-assured test. Since we cannot test whether the application server has started, we have to just wait an arbitrary amount of time.

Conclusion

This article has demonstrated several ways of writing integration tests with the Payara server using the Arquillian Cube extension. The next article will demonstrate two other useful extensions: Drone and Graphene. Stay tuned!