Reading Time: 9 minutes

Automate the generation of your unit tests in Java by using a testing plugin powered by Artificial Intelligence – AI. I’ll cover a new type of unit testing called unit regression testing and how AI can automatically generate them for your Java projects.

The Challenges with Java Unit Testing

I think we can all agree that writing Java JUnit tests comes at a cost. It takes a lot of time to write unit tests and then maintain them, right? It’s not so much those valuable unit tests that validate complex business logic but rather those mundane and exhaustive utility tests that we try to settle on a sweet spot to cover.

We spend an enormous amount of time writing unit tests which translates to significant costs for companies. Did you know that the average company spends about 14K per developer/year on this! This is because we know the value of catching errors early in development as opposed to operations. However, in order to start benefiting from this, we have to write an army of unit tests which must be continuously maintained, a substantial effort.

Most companies aim for 60%-80% code coverage

How many times have you convinced yourself that you don’t have time to write every single permutation of a Class’s methods? I bet you identify the highest value tests to write and settle on that – totally normal. It’s a balancing act at times.

Java Unit Regressions Tests Using AI | Benefits

Artificial Intelligence has been helping about every industry out there. It’s about time that AI starts helping us Java programmers .. some respect please! AI can help by automatically generating and maintaining a new breed of unit tests called “unit regression tests“.

Wait up here, are you saying that we don’t need to write any more unit tests in Java? No, I’m not saying that. Hear me out.

As of today, AI is not able to compete with a normal human generating complex business logic unit tests or know the intent of your code (is it logically correct).

You will still have to write unit tests. I’m talking about the one’s you know have the highest value and you couldn’t sleep at night knowing that you didn’t commit them with the source code …. yeah those! You are a normal human, right?!

What AI can really excel at is in supporting us in automatically writing our unit tests for …

Those mundane and exhaustive utility and boilerplate tests we have no time for
Quickly Increasing code coverage by an additional~20%
Generating corner and edge cases we may miss
Writing an army of unit regression tests that span in breadth and depth
Identifying and generating new unit tests that have a dependency on an external Class which you might miss

In essence, AI will support our human developers who are still identifying and writing high value unit tests. These new AI generated “unit regression tests” serve as an incredible safety net and time/cost saver.

We can move ahead with greater confidence when changes are made, especially when we find our team is strapped for time or resources. This is also a huge cost savings as the AI generated unit regression tests are constantly maintained on every commit you make. The tests are always up to date.

How to use AI to Generate Automated Regression Unit Tests

There are several companies out there that have targeted this new emerging space/technology, however I believe the leader to be a company by the name of DiffBlue. Their grassroots stem from a group of world leading experts from Oxford university specializing in software verification.

They recently came out with a community edition (CE) IntelliJ plugin for their Diffblue Cover product. I’m going to show you just how easy it is to use this IntelliJ plugin to create an army of unit regression tests for your Java application.

I start off nice and simple and then I’ll address other scenarios and environments in subsequent articles. You’ll be happy to know that these tests are human readable!

Want to Check out the YouTube version? Check it out here ..

How Diffblue uses AI to Automate Unit Testing for Java Applications | IntelliJ Plugin

I recommend you use my Docker image which has a Java developer environment setup in Ubuntu with the IntelliJ IDEA ready for you to experiment with. Check out my article here to get your environment setup with one command. Let’s go ..

At this point, You should have the IntelliJ IDEA up and running either through my Docker container or your own environment. This plugin works with either Maven or Gradle and Java 8 or 11. JUnit is also a required dependency in your Java application (supports JUnit 4 and JUnit 5).

Step 1: We need to install the Diffblue Cover CE plugin. Click on the “Install” button where you see my mouse arrow in on and then press “OK” button once the download is complete. You will have to shutdown IntelliJ or re-run the container after install (you will be prompted).

Step 2: Now that you have re-started the IDE, let’s get a sample Java application into IntelliJ.

You will have to “Get from VCS” and enter the following GitHub link https://github.com/diffblue/CoreBanking like so ..

Your Directory path will be different of course, my real name is not MVP Java, it’s andy …shhh, don’t tell anyone ;

Right now, we have no tests written which is of course a perfect fit for our very first example. as you can see below, there is not test directory … yet!

Diffblue sample project for AI unit regression testing | MVP Java

This could be a great use case for projects that never had any unit tests ever written. You might think I am exaggerating but I assure you that I am not. I have worked on many legacy applications in which we had no tests, no safety net and the risk of regressions was very high for each new deployment. In fact, it was always factored into the risk model (we put our thumb to the wind). I wish I had this plugin back in the day.

In a matter of minutes, we’ll get an army of unit regression tests that will represent the current state of the code and will constantly be updated as we change that code.

Step 3: Generating AI Java JUnit tests on a single Class

We have the option to create AI generated unit regression tests for either …

A single method within a Class
All methods in a Class
Entire Project

Let’s go with all methods in the “Account” Class so we don’t get buried under an ocean of unit tests for our first example.

You can either right-click on the “Account” Class in the Projects view or the “Account” Tab or the Class name in the source code (whatever you like) and then select “Write Tests”

Off it goes! It is important to ensure that before invoking the Diffblue plugin that your Java application must fully compile from root and run! The plugin builds your project and then will analyze the Java bytecode as well as the source code files in question. In this scenario, I get the following message as the last line in the IntelliJ Event Log window ..

Test creation complete: Created 12 tests in total for 8 methods from source class Account

1	Test creation complete: Created 12 tests in total for 8 methods from source class Account

The following test folder gets created with the auto-generated AI unit regression tests ..

Once the regression tests are auto-generated, you will prompted if you want to commit the “AccountTest” file to our GitHub repository. For now, answer “Cancel”.

I was surprised at how human readable the tests are, best I’ve seen yet!

You won’t see garbage tests or tests that offer little value either. These are not statically analyzed like a plain code generator would do – just pumping out templates. There are advanced machine learning algorithms working here.

Check them out below (written with JUnit4 in this example) …

package io.diffblue.corebanking.account;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;

import io.diffblue.corebanking.client.Client;
import io.diffblue.corebanking.transaction.CashTransaction;

import java.util.Date;

import org.junit.Rule;

import org.junit.Test;
import org.junit.rules.ExpectedException;

public class AccountTest {
    @Rule
    public ExpectedException thrown = ExpectedException.none();

    @Test
    public void testConstructor() {
        Account actualAccount = new Account(1234567890L, new Client("Dr Jane Doe"), 10L);
        assertEquals(
                "Account: | Acc #: 1234567890\t | Acc name: Current\t | Acc holder: Dr Jane Doe\t | Acc balance: 10\t | Acc"
                        + " state: OPEN\t |\n" + "Account statement empty.",
                actualAccount.toString());
        assertEquals(1234567890L, actualAccount.getAccountNumber());
        assertEquals("Current", actualAccount.getAccountName());
        assertEquals(Account.AccountState.OPEN, actualAccount.getAccountState());
        assertEquals(10L, actualAccount.getCurrentBalance());
        assertEquals("Account statement empty.", actualAccount.getAccountStatement().toString());
    }

    @Test
    public void testConstructor2() {
        Account actualAccount = new Account(1234567890L, new Client("Dr Jane Doe"), 10L);
        assertEquals(
                "Account: | Acc #: 1234567890\t | Acc name: Current\t | Acc holder: Dr Jane Doe\t | Acc balance: 10\t | Acc"
                        + " state: OPEN\t |\n" + "Account statement empty.",
                actualAccount.toString());
        assertEquals(1234567890L, actualAccount.getAccountNumber());
        assertEquals("Current", actualAccount.getAccountName());
        assertEquals(Account.AccountState.OPEN, actualAccount.getAccountState());
        assertEquals(10L, actualAccount.getCurrentBalance());
        assertEquals("Account statement empty.", actualAccount.getAccountStatement().toString());
    }

    @Test
    public void testAddToBalance() throws AccountException {
        Account account = new Account(1234567890L, new Client("Dr Jane Doe"), 10L);
        account.addToBalance(10L);
        assertEquals(20L, account.getCurrentBalance());
    }

    @Test
    public void testAddToBalance2() throws AccountException {
        Account account = new Account(1234567890L, new Client("Dr Jane Doe"), 10L);
        account.addToBalance(10L);
        assertEquals(20L, account.getCurrentBalance());
    }

    @Test
    public void testTakeFromBalance() throws AccountException {
        Account account = new Account(1234567890L, new Client("Dr Jane Doe"), 10L);
        account.takeFromBalance(10L);
        assertEquals(0L, account.getCurrentBalance());
    }

    @Test
    public void testTakeFromBalance2() throws AccountException {
        thrown.expect(AccountException.class);
        (new Account(1234567890L, new Client("Dr Jane Doe"), 9223372036854775807L)).takeFromBalance(10L);
    }

    @Test
    public void testSetAccountName() throws AccountException {
        Account account = new Account(1234567890L, new Client("Dr Jane Doe"), 10L);
        account.setAccountName("Dr Jane Doe");
        assertEquals("Dr Jane Doe", account.getAccountName());
    }

    @Test
    public void testCloseAccount() throws AccountException {
        Account account = new Account(1234567890L, new Client("Dr Jane Doe"), 10L);
        account.closeAccount();
        assertEquals(Account.AccountState.CLOSED, account.getAccountState());
    }

    @Test
    public void testAddTransaction() throws AccountException {
        Account account = new Account(1234567890L, new Client("Dr Jane Doe"), 10L);
        Date date = new Date(1L);
        account.addTransaction(new CashTransaction(10L, date, new Account(1234567890L, new Client("Dr Jane Doe"), 10L)));
        assertEquals(1, account.getAccountStatement().getTransactions().size());
    }

    @Test
    public void testToString() {
        assertEquals(
                "Account: | Acc #: 1234567890\t | Acc name: Current\t | Acc holder: Dr Jane Doe\t | Acc balance: 10\t | Acc"
                        + " state: OPEN\t |\n" + "Account statement empty.",
                (new Account(1234567890L, new Client("Dr Jane Doe"), 10L)).toString());
    }

    @Test
    public void testEquals() {
        assertFalse((new Account(1234567890L, new Client("Dr Jane Doe"), 10L)).equals("o"));
        assertFalse((new Account(1234567890L, new Client("Dr Jane Doe"), 10L)).equals(null));
    }

    @Test
    public void testEquals2() {
        Account account = new Account(1234567890L, new Client("Dr Jane Doe"), 10L);
        assertTrue(account.equals(new Account(1234567890L, new Client("Dr Jane Doe"), 10L)));
    }

    @Test
    public void testEquals3() {
        Account account = new Account(3L, new Client("Dr Jane Doe"), 10L);
        assertFalse(account.equals(new Account(1234567890L, new Client("Dr Jane Doe"), 10L)));
    }
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

package io.diffblue.corebanking.account;

import static org.junit.Assert.assertEquals;

import static org.junit.Assert.assertFalse;

import static org.junit.Assert.assertTrue;

import io.diffblue.corebanking.client.Client;

import io.diffblue.corebanking.transaction.CashTransaction;

import java.util.Date;

import org.junit.Rule;

import org.junit.Test;

import org.junit.rules.ExpectedException;

public class AccountTest {

@Rule

public ExpectedException thrown = ExpectedException.none();

@Test

public void testConstructor() {

Account actualAccount = new Account(1234567890L, new Client("Dr Jane Doe"), 10L);

assertEquals(

+ " state: OPEN\t |\n" + "Account statement empty.",

actualAccount.toString());

assertEquals(1234567890L, actualAccount.getAccountNumber());

assertEquals("Current", actualAccount.getAccountName());

assertEquals(Account.AccountState.OPEN, actualAccount.getAccountState());

assertEquals(10L, actualAccount.getCurrentBalance());

assertEquals("Account statement empty.", actualAccount.getAccountStatement().toString());

}

@Test

public void testConstructor2() {

Account actualAccount = new Account(1234567890L, new Client("Dr Jane Doe"), 10L);

assertEquals(

+ " state: OPEN\t |\n" + "Account statement empty.",

actualAccount.toString());

assertEquals(1234567890L, actualAccount.getAccountNumber());

assertEquals("Current", actualAccount.getAccountName());

assertEquals(Account.AccountState.OPEN, actualAccount.getAccountState());

assertEquals(10L, actualAccount.getCurrentBalance());

assertEquals("Account statement empty.", actualAccount.getAccountStatement().toString());

}

@Test

public void testAddToBalance() throws AccountException {

Account account = new Account(1234567890L, new Client("Dr Jane Doe"), 10L);

account.addToBalance(10L);

assertEquals(20L, account.getCurrentBalance());

}

@Test

public void testAddToBalance2() throws AccountException {

Account account = new Account(1234567890L, new Client("Dr Jane Doe"), 10L);

account.addToBalance(10L);

assertEquals(20L, account.getCurrentBalance());

}

@Test

public void testTakeFromBalance() throws AccountException {

Account account = new Account(1234567890L, new Client("Dr Jane Doe"), 10L);

account.takeFromBalance(10L);

assertEquals(0L, account.getCurrentBalance());

}

@Test

public void testTakeFromBalance2() throws AccountException {

thrown.expect(AccountException.class);

(new Account(1234567890L, new Client("Dr Jane Doe"), 9223372036854775807L)).takeFromBalance(10L);

}

@Test

public void testSetAccountName() throws AccountException {

Account account = new Account(1234567890L, new Client("Dr Jane Doe"), 10L);

account.setAccountName("Dr Jane Doe");

assertEquals("Dr Jane Doe", account.getAccountName());

}

@Test

public void testCloseAccount() throws AccountException {

Account account = new Account(1234567890L, new Client("Dr Jane Doe"), 10L);

account.closeAccount();

assertEquals(Account.AccountState.CLOSED, account.getAccountState());

}

@Test

public void testAddTransaction() throws AccountException {

Account account = new Account(1234567890L, new Client("Dr Jane Doe"), 10L);

Date date = new Date(1L);

account.addTransaction(new CashTransaction(10L, date, new Account(1234567890L, new Client("Dr Jane Doe"), 10L)));

assertEquals(1, account.getAccountStatement().getTransactions().size());

}

@Test

public void testToString() {

assertEquals(

+ " state: OPEN\t |\n" + "Account statement empty.",

(new Account(1234567890L, new Client("Dr Jane Doe"), 10L)).toString());

}

@Test

public void testEquals() {

assertFalse((new Account(1234567890L, new Client("Dr Jane Doe"), 10L)).equals("o"));

assertFalse((new Account(1234567890L, new Client("Dr Jane Doe"), 10L)).equals(null));

}

@Test

public void testEquals2() {

Account account = new Account(1234567890L, new Client("Dr Jane Doe"), 10L);

assertTrue(account.equals(new Account(1234567890L, new Client("Dr Jane Doe"), 10L)));

}

@Test

public void testEquals3() {

Account account = new Account(3L, new Client("Dr Jane Doe"), 10L);

assertFalse(account.equals(new Account(1234567890L, new Client("Dr Jane Doe"), 10L)));

}

Step 4: Running the tests.

Right-Click on the “AccountTest” Class and run the tests via selecting “Run AccountTest”. You will get the following results …

We now have a set of unit regression tests in which we can commit and have as a safety net moving forward.

Catching Regressions with AI Unit Regression Tests

So now that we are feeling more confident in changing our code for the Account Class, let’s intentionally cause a regression and see if our new safety net saves us.

Let’s pretend that a developer modified the takeFromBalance method and incorrectly modify the last line from”-=” to “=-” … nasty!

  public void takeFromBalance(final long amount) throws AccountException {
    if (getAccountState() != AccountState.OPEN) {
      throw new AccountException("Cannot take from balance, account is closed.");
    }
    if (currentBalance + amount < 0) {
      throw new AccountException(
          "Trying to take "
              + amount
              + " from the existing balance of "
              + currentBalance
              + ". Not enough funds.");
    }
    currentBalance =- amount;
  }

public void takeFromBalance(final long amount) throws AccountException {

if (getAccountState() != AccountState.OPEN) {

throw new AccountException("Cannot take from balance, account is closed.");

}

if (currentBalance + amount < 0) {

throw new AccountException(

"Trying to take "

+ amount

+ " from the existing balance of "

+ currentBalance

+ ". Not enough funds.");

}

currentBalance =- amount;

}

Re-build and then right-click on the “Account” Class again to re-generate the tests via “Write Tests“. This is where it starts getting interesting 🙂

New regression tests have been auto generated in order to unit test the current behavior of the Java application. However, the old one’s (our safety net) is still there to make sure we didn’t break anything – a regression.

Once we re-run the entire suite of tests, we see that test testTakeFromBalance fails and this is great! Now we can go in there and see if this is an expected new behavior. This is where AI generated tests may not know that this is a new intended behavior, so we check out the test results and decide.

If this is a new intended behavior, then go ahead and remove this test and replace it with your own human generated test. If not, you are fortunate that this was caught so early and didn’t make it to production! This new type of testing (regression unit tests) is what is responsible for a term called “Shift Left Testing”.

Traditional, regression tests exercise end-to-end functional type tests containing external dependencies, services, storage and networking. This would dramatically slow down the entire code-build-test cycle that developers strive to quickly iterate over. We push these tests more to the right or down the CI pipeline.

What is happening with our AI generated regression tests is that dependencies are mocked out and can run earlier (or to the left) in the cycle. This is a huge advantage in finding bugs earlier (to the left) and that at the end of the day speeds up our time to market, improves the quality of our product and reduces cost.

What’s Next for Java AI Generated Tests

I have of course just scratched the surface in regard to the different scenarios and environments in which we can take advantage of these AI auto-generated unit tests.

I didn’t want to focus on the plugin options in this post however I will in later posts. You’ll be happy to know Mocking, Spring, Spring Boot are supported and that we can perform operations using a CLI interface. Of course, the best is yet to come as you will want to integrate this with your VCS like git and incorporate this in your CI pipeline.

This area is new and I think it will not only rapidly improve but expand to other areas in our software. It’s a tool we can certainly leverage. I hope you see how powerful this is, especially when used in tandem with your own unit tests.

In the end, I miss my test user “Foo Bar” and account id “111” but hey, I’m just human 😉

Share this post:

Twitter Facebook Reddit Email