Using Arquillian Cube to test containers in Kubernetes

This is a very quick note from a recent project where I wanted to run an integration test for an Apache Camel / Spring Boot application, running on OpenShift.

The question is - how do you test that a container is deployed succesfully into OpenShift or Kubernetes, especially if it has dependencies on other containers?

You can do this by writing an integration test using Arquillian Cube, the container testing framework. There are many ways to configure Cube (read the docs), including auto-scanning and hooking up with the fabric8-maven-plugin.

But, if like me, you want a bit less “magic”, and more explicit set-up for your test, then here’s what I used:

Add the following to your maven pom.xml - this will pull in Arquillian Cube:

<!-- Arquillian for integration (black-box) testing -->
<dependency>
  <groupId>org.arquillian.cube</groupId>
  <artifactId>arquillian-cube-openshift</artifactId>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>io.fabric8</groupId>
  <artifactId>kubernetes-assertions</artifactId>
  <scope>test</scope>
</dependency>

Next, create a file src/test/resources/arquillian.xml - this will configure Arquillian to use an existing namespace, my-itest-namespace, deploy some Kubernetes definitions (YAML files) you’ve defined for your application in myapplication.yml and then deploy any dependent objects, also defined in database.yml, message-broker.yml and so on.

<arquillian xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xmlns="http://jboss.org/schema/arquillian"
            xsi:schemaLocation="http://jboss.org/schema/arquillian
               http://jboss.org/schema/arquillian/arquillian_1_0.xsd">

  <!-- This file describes the resources that need to be created in OpenShift to run a successful test -->
  <extension qualifier="kubernetes">
    <property name="namespace.use.existing">my-itest-namespace</property>
    <property name="env.config.resource.name">myapplication.yml</property>

    <!-- Add YAMLs for any other OpenShift/K8s resources in here -->
    <property name="env.dependencies">
      file:src/test/resources/database.yml
      file:src/test/resources/message-broker.yml
      file:src/test/resources/auth-server.yml
      file:src/test/resources/another-microservice.yml
    </property>
  </extension>
</arquillian>

Define the YAML files for Kubernetes/OpenShift resources for your application - e.g. DeploymentConfigs, Routes, Services and so on - inside src/test/resources/myapplication.yml.

Also define YAML files for any dependent containers, like database, message broker or auth server, and place them in src/main/resources/xxx.yml, as given in the example.

Finally, write a unit test class which checks that the deployment happens successfully:

package com.example;

import io.fabric8.kubernetes.client.KubernetesClient;
import org.jboss.arquillian.container.test.api.RunAsClient;
import org.jboss.arquillian.junit.Arquillian;
import org.jboss.arquillian.test.api.ArquillianResource;
import org.junit.Test;
import org.junit.runner.RunWith;

import static io.fabric8.kubernetes.assertions.Assertions.assertThat;

@RunWith(Arquillian.class)
@RunAsClient
public class MyIntegrationTestIT {

    @ArquillianResource
    KubernetesClient client;

    @Test
    public void testAppProvisionsRunningPods() throws Exception {
        assertThat(client)
            .deployment("myapp")
            .pods().isPodReadyForPeriod();
    }
}

You can now run your test using:

mvn clean test -Dtest=*IT

This test will wait for the Deployment myapp to run successfully and for the pods to come into the Ready state.

Then it’s just a matter of extending this test, to do things like:

• check whether the pod comes up correctly
• fire requests to your application’s REST endpoint and see whether it behaves as expected

…and adding this test into a CI/CD pipeline, to make sure your containers are integration tested automatically. Neat.