DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Enterprise AI Trend Report: Gain insights on ethical AI, MLOps, generative AI, large language models, and much more.

2024 Cloud survey: Share your insights on microservices, containers, K8s, CI/CD, and DevOps (+ enter a $750 raffle!) for our Trend Reports.

PostgreSQL: Learn about the open-source RDBMS' advanced capabilities, core components, common commands and functions, and general DBA tasks.

AI Automation Essentials. Check out the latest Refcard on all things AI automation, including model training, data security, and more.

Related

  • Maven Plugin Testing - in a Modern Way - Part I
  • Creating Test Stages With JUnit
  • The Most Popular Technologies for Java Microservices Right Now
  • Micronaut With Relation Database and...Tests

Trending

  • Integration of AI Tools With SAP ABAP Programming
  • ChatGPT Code Smell [Comic]
  • Securing Cloud Storage Access: Approach to Limiting Document Access Attempts
  • Secure Your API With JWT: Kong OpenID Connect
  1. DZone
  2. Testing, Deployment, and Maintenance
  3. Testing, Tools, and Frameworks
  4. JUnit Test Groups for More Reliable Development

JUnit Test Groups for More Reliable Development

By 
Horatiu Dan user avatar
Horatiu Dan
·
Nov. 19, 20 · Tutorial
Like (4)
Save
Tweet
Share
7.0K Views

Join the DZone community and get the full member experience.

Join For Free

Introduction

As a product is being developed and maintained its test suite is enriched as features and functionalities are added and enhanced. Ideally, development teams should aim having a lot of quick unit tests that are run whenever modifications in the code are made. It is great if these are written before or at least together with the tested code, cover as many use cases as possible and finish running after a reasonable amount of time. By all means, a reasonable amount of time is an entity difficult to quantify. 

On the other hand, there are a lot of live products that solve quite complex business problems whose thorough verification concerns cross-cutting through multiple application layers at once and thus, the number of integration tests is significant. Running all these unit and integration tests whenever the code is touched is not always feasible, as the productivity and the development speed is decreased considerably. Again, productivity and development speed are hard to quantify but should be always traded for correctness. 

Under such circumstances, developers need to imagine possible compromises that help delivering confidently and reliably. One such compromise is the ability to split quick and small unit tests from the heavier integration tests. 

This post is going to present a small working example that describes how to split tests into multiple categories (groups) and accomplish the above mentioned compromise.

Context

‘testcategories’ is a small project especially developed for the purpose of this post, using the following: 

  • Java 11
  • Apache Maven 3.6.3
  • JUnit 4.13
  • Maven Surefire Plugin 2.22.0

It contains two classes each simulating the retrieval of a certain category of entities available from two different sources. They are great sports people, either Romanian or international.

Java 
xxxxxxxxxx
1
21
 
1 
public class Local {
2 
    
3 
    public List<String> champions() {
4 
        System.out.println("Retrieving local champions...");
5 
        return List.of("Simona Halep", "Cristina Neagu");
6 
    }
7 
}
8
9 
public class Remote {
10
11 
    public List<String> champions() {
12 
        try {
13 
            System.out.println("Retrieving remote champions...");
14 
            Thread.sleep(5000);
15 
            
16 
            return List.of("Roger Federer", "Dan Carter");
17 
        } catch (InterruptedException e) {
18 
            throw new RuntimeException("Champions unavailable.");
19 
        }
20 
    }
21 
}


For the sake of this example, it is imagined that the ‘local’ ones are easy to reach, cheaper to retrieve in terms of the involved resources, while the ‘remote’ ones more expensive. This is simulated by forcing the designated method to last at least 5 seconds, by putting the current thread to sleep. 

Under these circumstances, it is reasonable to state that Local#champions() method could be unit tested, while the Remote#champions() one integration tested. 

Solutions

In order to split unit and integration tests into distinct categories, there are at least two approaches. Both may be accomplished with the help of ‘maven-surefire-plugin’ by configuring it accordingly. 

  • Adopt a naming convention for the test classes, depending on the test category they belong to and configure Maven Surefire Plugin so that it includes / excludes what is executed in each scenario. 
XML 
xxxxxxxxxx
1
12
 
1 
<plugin>
2 
    <artifactId>maven-surefire-plugin</artifactId>
3 
    <version>2.22.0</version>
4 
    <configuration>
5 
        <includes>
6 
            <include>**/*Test.java</include>
7 
        </includes>
8 
        <excludes>            
9 
            <exclude>**/*Integration.java</exclude>
10 
        </excludes>
11 
    </configuration>
12 
</plugin>


It is expected that classes whose names end in ‘Test’ will designate unit tests, while those suffixed with ‘Integration’, integration tests. 

I have experimented this solution before, it’s been working great for years, the only drawback I find is that one cannot have both unit and integration tests residing in the same class. 

  • Annotate classes or tests inside a class with JUnit’s @Category and categorize all or a single test as being part of a certain group designated by a custom marker interface. Moreover, configure Maven Surefire Plugin to run the desired group(s) of tests.
XML 
xxxxxxxxxx
1
 
1 
<plugin>
2 
    <groupId>org.apache.maven.plugins</groupId>
3 
    <artifactId>maven-surefire-plugin</artifactId>
4 
    <version>2.22.0</version>
5 
    <configuration>                      
6 
        <groups>com.hcd.testcategories.config.UnitTests</groups>
7 
    </configuration>
8 
</plugin>


As part of this post, the latter solution is detailed. 

Split Unit and Integration Test

The sample project contains 5 tests that are organized in 4 different classe so that multiple scenarios can be presented. In order to split them, two categories are defined as two marker interfaces, whose names were chosen so that it is clear what they designate. 

Java 
xxxxxxxxxx
1
 
1 
public interface UnitTests {
2
3 
}
4
5 
public interface IntegrationTests {
6
7 
}


In order for a class to be considered part of a group, it is enough to annotate it with @Category and provide as value the corresponding marker interface – @Category(UnitTests.class). 

Moreover, one can put a test into multiple groups by providing the value as comma separated marker interfaces – @Category({SlowTests.class, DatabaseTests.class}). 

Nevertheless, in the example illustrated here, it is a nonsense to consider a test both as unit and integration in the same time. 

The mentioned tests are detailed below. 

  • LocalTest – contains 1 unit test – class is annotated with @Category(UnitTests.class) 
Java 
xxxxxxxxxx
1
12
 
1 
@Category(UnitTests.class)
2 
public class LocalTest {
3
4 
    @Test  
5 
    public void champions() {
6 
        final Local local = new Local();        
7 
        List<String> result = local.champions();
8 
        
9 
        assertEquals(2, result.size());
10 
        assertTrue(result.contains("Simona Halep"));
11 
    }
12 
}


  • RemoteTest – contains 1 integration test – class is annotated with @Category(IntegrationTests.class)
Java
  




xxxxxxxxxx
1
12


 
1 
@Category(IntegrationTests.class)
2 
public class RemoteTest {
3
4 
    @Test
5 
    public void champions() {
6 
        final Remote remote = new Remote();    
7 
        List<String> result = remote.champions();
8 
        
9 
        assertEquals(2, result.size());
10 
        assertTrue(result.contains("Roger Federer"));
11 
    }
12 
}



  • MixedTest – contains 1 unit and 1 integration test – class is not annotated, but each test is annotated with @Category(UnitTests.class) and @Category(IntegrationTests.class) respectively 
Java 
xxxxxxxxxx
1
22
 
1 
public class MixedTest {
2
3 
    @Test
4 
    @Category(UnitTests.class)
5 
    public void localChampions() {
6 
        final Local local = new Local();        
7 
        List<String> result = local.champions();
8 
        
9 
        assertEquals(2, result.size());
10 
        assertTrue(result.contains("Simona Halep"));
11 
    }
12 
    
13 
    @Test
14 
    @Category(IntegrationTests.class)
15 
    public void remoteChampions() {
16 
        final Remote remote = new Remote();    
17 
        List<String> result = remote.champions();
18 
        
19 
        assertEquals(2, result.size());
20 
        assertTrue(result.contains("Roger Federer"));
21 
    }
22 
}


  • NewLocalTest – contains 1 unit test – neither the class, nor the test is annotated
Java 
xxxxxxxxxx
1
11
 
1 
public class NewLocalTest {
2
3 
    @Test  
4 
    public void champions() {
5 
        final Local local = new Local();        
6 
        List<String> result = local.champions();
7 
        
8 
        assertEquals(2, result.size());
9 
        assertTrue(result.contains("Cristina Neagu"));
10 
    }
11 
}


The intention with the last one is to designate a test for which the developer has forgotten to annotate it and thus, accidentally left it out of any of the two categories (groups). Such a situation is very likely to appear in real life, after the point the existing tests have been split. 

Maven Configuration

In order to be easier to run the tests in various scenarios, one can define a Maven property that designates the test categories (groups) taken into account at a certain execution. Moreover, a couple of profiles can be used, so that it is easier to invoke the build / test command. 

Below is the complete POM file of the sample project. 

XML 
xxxxxxxxxx
1
86
 
1 
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
2 
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
3
4 
    <modelVersion>4.0.0</modelVersion>
5
6 
    <groupId>com.hcd</groupId>
7 
    <artifactId>testcategories</artifactId>
8 
    <version>0.0.1-SNAPSHOT</version>
9 
    <description>Sample Maven Project that differentiate between Unit and Integration tests at runtime.</description>
10 
  
11 
    <properties>      
12 
        <test.categories/>
13 
    </properties>
14 
  
15 
    <profiles>
16 
        <profile>
17 
            <id>all-tests</id>          
18 
            <activation>
19 
                <activeByDefault>true</activeByDefault>
20 
            </activation>
21 
            <properties>
22 
                <build.profile.id>all-tests</build.profile.id>
23 
                <test.categories/>
24 
            </properties>
25 
        </profile>
26 
        <profile>
27 
            <id>unit-tests</id>        
28 
            <properties>
29 
                <build.profile.id>unit-tests</build.profile.id>
30 
                <test.categories>com.hcd.testcategories.config.UnitTests</test.categories>
31 
            </properties>
32 
        </profile>
33 
        <profile>
34 
            <id>integration-tests</id>
35 
            <properties>
36 
                <build.profile.id>integration-tests</build.profile.id>
37 
                <test.categories>com.hcd.testcategories.config.IntegrationTests</test.categories>
38 
            </properties>
39 
        </profile>
40 
    </profiles>
41
42 
    <build>
43 
        <pluginManagement>
44 
            <plugins>
45 
                <plugin>
46 
                    <groupId>org.apache.maven.plugins</groupId>
47 
                    <artifactId>maven-compiler-plugin</artifactId>  
48 
                    <version>3.8.0</version>
49 
                    <configuration>
50 
                        <release>11</release>
51 
                        <includeEmptyDirs>true</includeEmptyDirs>
52 
                    </configuration>                  
53 
                </plugin>            
54 
                <plugin>
55 
                    <groupId>org.apache.maven.plugins</groupId>
56 
                    <artifactId>maven-surefire-plugin</artifactId>
57 
                    <version>2.22.0</version>
58 
                    <configuration>                      
59 
                        <groups>${test.categories}</groups>
60 
                    </configuration>
61 
                </plugin>
62 
            </plugins>
63 
        </pluginManagement>
64 
        
65 
        <plugins>
66 
            <plugin>
67 
                <groupId>org.apache.maven.plugins</groupId>
68 
                <artifactId>maven-compiler-plugin</artifactId>
69 
            </plugin>
70 
            <plugin>
71 
                <groupId>org.apache.maven.plugins</groupId>
72 
                <artifactId>maven-surefire-plugin</artifactId>
73 
            </plugin>
74 
        </plugins>
75 
    </build>
76 
        
77 
    <dependencies>
78 
        <dependency>
79 
        <groupId>junit</groupId>
80 
            <artifactId>junit</artifactId>
81 
            <version>4.13</version>
82 
            <scope>test</scope>
83 
        </dependency>
84 
    </dependencies>
85 
</project>
85 
</project>


The profiles defined are: 

  • ‘all-tests’ – default profile, executes all unit, integration and non-categorized tests. test.categories property is empty.
  • ‘unit-tests’ – executes all annotated unit tests. test.categories = com.hcd.testcategories.config.UnitTests.
  • ‘integration-tests’ – executes all annotated integration tests. test.categories = com.hcd.testcategories.config.IntegrationTests.

Running Scenarios

  • Run all defined tests – unit, integration and non-categorized

  • by invoking the default profile implicitly

> mvn clean test 

  • by invoking the ‘all-tests’ profile 

> mvn clean test -Pall-tests 

  • by specifying an empty ‘test.categories’ property 

> mvn clean test -Dtest.categories=

All 5 tests are executed. The output is below. 

PowerShell 
xxxxxxxxxx
1
27
 
1 
[INFO] -------------------------------------------------------
2 
[INFO]  T E S T S
3 
[INFO] -------------------------------------------------------
4 
[INFO] Running com.hcd.testcategories.LocalTest
5 
Retrieving local champions...
6 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.107 s - in com.hcd.testcategories.LocalTest
7 
[INFO] Running com.hcd.testcategories.MixedTest
8 
Retrieving remote champions...
9 
Retrieving local champions...
10 
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.959 s - in com.hcd.testcategories.MixedTest
11 
[INFO] Running com.hcd.testcategories.NewLocalTest
12 
Retrieving local champions...
13 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.007 s - in com.hcd.testcategories.NewLocalTest
14 
[INFO] Running com.hcd.testcategories.RemoteTest
15 
Retrieving remote champions...
16 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.964 s - in com.hcd.testcategories.RemoteTest
17 
[INFO]
18 
[INFO] Results:
19 
[INFO]
20 
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0
21 
[INFO]
22 
[INFO] ------------------------------------------------------------------------
23 
[INFO] BUILD SUCCESS
24 
[INFO] ------------------------------------------------------------------------
25 
[INFO] Total time:  14.651 s
26 
[INFO] Finished at: 2020-11-12T12:21:31+02:00
27 
[INFO] ------------------------------------------------------------------------


  •  Run all annotated tests – unit and integration

  • by specifying the groups (comma separated) as values for the ‘test.categories’ property

> mvn clean test -Dtest.categories=com.hcd.testcategories.config.UnitTests,com.hcd.testcategories.config.IntegrationTests

All 4 annotated tests are executed. The non-annotated one in NewLocalTest is not. The output is below. 

PowerShell 
xxxxxxxxxx
1
25
 
1 
[INFO] -------------------------------------------------------
2 
[INFO]  T E S T S
3 
[INFO] -------------------------------------------------------
4 
[INFO] Running com.hcd.testcategories.LocalTest
5 
Retrieving local champions...
6 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.014 s - in com.hcd.testcategories.LocalTest
7 
[INFO] Running com.hcd.testcategories.MixedTest
8 
Retrieving remote champions...
9 
Retrieving local champions...
10 
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.942 s - in com.hcd.testcategories.MixedTest
11 
[INFO] Running com.hcd.testcategories.RemoteTest
12 
Retrieving remote champions...
13 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.986 s - in com.hcd.testcategories.RemoteTest
14 
[INFO]
15 
[INFO] Results:
16 
[INFO]
17 
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
18 
[INFO]
19 
[INFO] ------------------------------------------------------------------------
20 
[INFO] BUILD SUCCESS
21 
[INFO] ------------------------------------------------------------------------
22 
[INFO] Total time:  14.580 s
23 
[INFO] Finished at: 2020-11-12T13:08:21+02:00
24 
[INFO] ------------------------------------------------------------------------


  • Run all annotated unit tests

  • by invoking the ‘unit-tests’ profile

> mvn clean test -Punit-tests

  • by specifying the unit tests group as a value for the ‘test.categories’ property 

> mvn clean test -Dtest.categories=com.hcd.testcategories.config.UnitTests 

Both tests that are in the custom UnitTests category are executed – one from the LocalTest annotated class and one from the non-anotated MixedTest class, but annotated at method level. The output is below. 

PowerShell 
xxxxxxxxxx
1
21
 
1 
[INFO] -------------------------------------------------------
2 
[INFO]  T E S T S
3 
[INFO] -------------------------------------------------------
4 
[INFO] Running com.hcd.testcategories.LocalTest
5 
Retrieving local champions...
6 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.02 s - in com.hcd.testcategories.LocalTest
7 
[INFO] Running com.hcd.testcategories.MixedTest
8 
Retrieving local champions...
9 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.008 s - in com.hcd.testcategories.MixedTest
10 
[INFO]
11 
[INFO] Results:
12 
[INFO]
13 
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
14 
[INFO]
15 
[INFO] ------------------------------------------------------------------------
16 
[INFO] BUILD SUCCESS
17 
[INFO] ------------------------------------------------------------------------
18 
[INFO] Total time:  5.891 s
19 
[INFO] Finished at: 2020-11-12T13:27:19+02:00
20 
[INFO] ------------------------------------------------------------------------


  • Run all annotated integration tests

  • by invoking the ‘integration-tests’ profile

> mvn clean test -Pintegration-tests 

  • by specifying the integration tests group as a value for the ‘test.categories’ property

> mvn clean test -Dtest.categories=com.hcd.testcategories.config.IntegrationTests 

Both tests that are in the custom IntegrationTests category are executed – one from the RemoteTest annotated class and one from the non-anotated MixedTest class, but annotated at method level. The output is below. 

PowerShell
x
21
 
1 
[INFO] -------------------------------------------------------
2 
[INFO]  T E S T S
3 
[INFO] -------------------------------------------------------
4 
[INFO] Running com.hcd.testcategories.MixedTest
5 
Retrieving remote champions...
6 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.993 s - in com.hcd.testcategories.MixedTest
7 
[INFO] Running com.hcd.testcategories.RemoteTest
8 
Retrieving remote champions...
9 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.967 s - in com.hcd.testcategories.RemoteTest
10 
[INFO]
11 
[INFO] Results:
12 
[INFO]
13 
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
14 
[INFO]
15 
[INFO] ------------------------------------------------------------------------
16 
[INFO] BUILD SUCCESS
17 
[INFO] ------------------------------------------------------------------------
18 
[INFO] Total time:  15.140 s
19 
[INFO] Finished at: 2020-11-12T13:46:22+02:00
20 
[INFO] ------------------------------------------------------------------------


Conclusions

  • The quickest execution is, without any doubt, the one that runs just the explicitly marked unit tests.
  • The most thorough execution is the one that runs all defined tests (irrespective of the defined groups / categories); it definitely takes longer, but it is the most reliable one in terms of confidence.
  • No written test will cease to run during the default execution if for example a developer forgets to annotate either the containing test class or the test method (see point 1. and NewLocalTest class in the previous section).
  • One should apply each of the presented scenarios, depending on the particular circumstance and the purpose of the aimed achievement. During development the former seems to be the proper candidate, while the latter is more suitable for pre-production builds.

Code

The sample project is available now in GitHub - testcategories

unit test Apache Maven Integration JUnit integration test

Published at DZone with permission of Horatiu Dan. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • Maven Plugin Testing - in a Modern Way - Part I
  • Creating Test Stages With JUnit
  • The Most Popular Technologies for Java Microservices Right Now
  • Micronaut With Relation Database and...Tests

Partner Resources

Comments

ABOUT US

  • About DZone
  • Send feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends: