Reading Time: 8 minutes

Writing Spring Boot Integration Tests takes time, or any type of test for that matter. What if we could delegate that task to a plugin that used AI to generate them for us?

Testing with Spring Boot

Although Spring Boot offers us a huge leap in productivity, we must still take the time in developing our army of tests. Writing all the Unit and Integration tests you want is probably not going to happen. Usually a trade off between time and budget are made and we end up settling on writing the highest value tests – or at least we think we are.

Help is now here to level the playing field where we can leverage AI (Artificial Intelligence) to auto generate all our Java Unit and Spring Integration tests. The benefits are extraordinary.

You get a huge spike in code coverage and an army of regression tests created for you in minutes! This gives us the confidence to move forward in making changes in our application knowing that we have a safety net of tests that will always run. The goal is to have these test catch regressions. But are these AI generated tests actually useful and how do they compare against developer written tests?

Spring Testing | Developer vs AI

I came across an IntelliJ plugin by DiffBlue which auto generates Java Unit and Spring Integration tests by using Artificial Intelligence – AI. I’ve already written a post on how to generate your Java Unit tests with it here. Probably best to read that one first. This post goes a step further and covers how to generate Spring Integration tests via the plugin.

The Spring Boot PetClinic application available on GitHub will be used to compare the developer vs AI generated tests. It already comes with Spring Boot Integration tests written so it will serve as our basis to compare the AI tests against.

If you want to check out the YouTube tutorial version of this post, check it out here ..

PetClinic is a Spring Boot Web MVC application and I’ve decided to focus on the MVC @Controller VetContoller class. First, we’ll take a look at the VetController source code and see what kind of developer tests where written in the VetContollerTests class.

After that, we’ll generate the DiffBlue AI powered integration tests and compare the results.

Spring MVC @Controller | Developer Written Integration Tests

The following MVC @Controller VetContoller class has 2 methods which have a mix of web service architectural styles. The first showVetList is a more traditional html rendering of the endpoint whereas the second method showResourcesVetList is more in line with your typical REST endpoint architectural style.

package org.springframework.samples.petclinic.vet;

import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.ResponseBody;
import org.springframework.web.bind.annotation.RestController;

import java.util.Map;

/**
 * @author Juergen Hoeller
 * @author Mark Fisher
 * @author Ken Krebs
 * @author Arjen Poutsma
 */
@Controller
class VetController {

	private final VetRepository vets;

	public VetController(VetRepository clinicService) {
		this.vets = clinicService;
	}

	@GetMapping("/vets.html")
	public String showVetList(Map<String, Object> model) {
		// Here we are returning an object of type 'Vets' rather than a collection of Vet
		// objects so it is simpler for Object-Xml mapping
		Vets vets = new Vets();
		vets.getVetList().addAll(this.vets.findAll());
		model.put("vets", vets);
		return "vets/vetList";
	}

	@GetMapping({ "/vets" })
	public @ResponseBody Vets showResourcesVetList() {
		// Here we are returning an object of type 'Vets' rather than a collection of Vet
		// objects so it is simpler for JSon/Object mapping
		Vets vets = new Vets();
		vets.getVetList().addAll(this.vets.findAll());
		return vets;
	}

}

package org.springframework.samples.petclinic.vet;

import org.springframework.stereotype.Controller;

import org.springframework.web.bind.annotation.GetMapping;

import org.springframework.web.bind.annotation.ResponseBody;

import org.springframework.web.bind.annotation.RestController;

import java.util.Map;

/**

* @author Juergen Hoeller

* @author Mark Fisher

* @author Ken Krebs

* @author Arjen Poutsma

@Controller

class VetController {

private final VetRepository vets;

public VetController(VetRepository clinicService) {

this.vets = clinicService;

}

@GetMapping("/vets.html")

public String showVetList(Map<String, Object> model) {

// Here we are returning an object of type 'Vets' rather than a collection of Vet

// objects so it is simpler for Object-Xml mapping

Vets vets = new Vets();

vets.getVetList().addAll(this.vets.findAll());

model.put("vets", vets);

return "vets/vetList";

}

@GetMapping({ "/vets" })

public @ResponseBody Vets showResourcesVetList() {

// Here we are returning an object of type 'Vets' rather than a collection of Vet

// objects so it is simpler for JSon/Object mapping

Vets vets = new Vets();

vets.getVetList().addAll(this.vets.findAll());

return vets;

}

The following VetContollerTests class contains the developer written Spring Boot Integration tests.

/*
 * Copyright 2012-2019 the original author or authors.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      https://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package org.springframework.samples.petclinic.vet;

import static org.mockito.BDDMockito.given;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.content;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.model;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.view;

import org.assertj.core.util.Lists;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest;
import org.springframework.boot.test.mock.mockito.MockBean;
import org.springframework.http.MediaType;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.ResultActions;

/**
 * Test class for the {@link VetController}
 */
@WebMvcTest(VetController.class)
class VetControllerTests {

	@Autowired
	private MockMvc mockMvc;

	@MockBean
	private VetRepository vets;

	@BeforeEach
	void setup() {
		Vet james = new Vet();
		james.setFirstName("James");
		james.setLastName("Carter");
		james.setId(1);
		Vet helen = new Vet();
		helen.setFirstName("Helen");
		helen.setLastName("Leary");
		helen.setId(2);
		Specialty radiology = new Specialty();
		radiology.setId(1);
		radiology.setName("radiology");
		helen.addSpecialty(radiology);
		given(this.vets.findAll()).willReturn(Lists.newArrayList(james, helen));
	}

	@Test
	void testShowVetListHtml() throws Exception {
		mockMvc.perform(get("/vets.html")).andExpect(status().isOk()).andExpect(model().attributeExists("vets"))
				.andExpect(view().name("vets/vetList"));
	}

	@Test
	void testShowResourcesVetList() throws Exception {
		ResultActions actions = mockMvc.perform(get("/vets").accept(MediaType.APPLICATION_JSON))
				.andExpect(status().isOk());

		actions.andExpect(content().contentType(MediaType.APPLICATION_JSON)).andExpect(status().isOk())
				.andExpect(jsonPath("$.vetList[0].id").value(1));
	}

}

* Licensed under the Apache License, Version 2.0 (the "License");

* you may not use this file except in compliance with the License.

* You may obtain a copy of the License at

* https://www.apache.org/licenses/LICENSE-2.0

* Unless required by applicable law or agreed to in writing, software

* distributed under the License is distributed on an "AS IS" BASIS,

* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

* See the License for the specific language governing permissions and

* limitations under the License.

package org.springframework.samples.petclinic.vet;

import static org.mockito.BDDMockito.given;

import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;

import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.content;

import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath;

import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.model;

import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;

import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.view;

import org.assertj.core.util.Lists;

import org.junit.jupiter.api.BeforeEach;

import org.junit.jupiter.api.Test;

import org.springframework.beans.factory.annotation.Autowired;

import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest;

import org.springframework.boot.test.mock.mockito.MockBean;

import org.springframework.http.MediaType;

import org.springframework.test.web.servlet.MockMvc;

import org.springframework.test.web.servlet.ResultActions;

/**

* Test class for the {@link VetController}

@WebMvcTest(VetController.class)

class VetControllerTests {

@Autowired

private MockMvc mockMvc;

@MockBean

private VetRepository vets;

@BeforeEach

void setup() {

Vet james = new Vet();

james.setFirstName("James");

james.setLastName("Carter");

james.setId(1);

Vet helen = new Vet();

helen.setFirstName("Helen");

helen.setLastName("Leary");

helen.setId(2);

Specialty radiology = new Specialty();

radiology.setId(1);

radiology.setName("radiology");

helen.addSpecialty(radiology);

given(this.vets.findAll()).willReturn(Lists.newArrayList(james, helen));

}

@Test

void testShowVetListHtml() throws Exception {

mockMvc.perform(get("/vets.html")).andExpect(status().isOk()).andExpect(model().attributeExists("vets"))

.andExpect(view().name("vets/vetList"));

}

@Test

void testShowResourcesVetList() throws Exception {

ResultActions actions = mockMvc.perform(get("/vets").accept(MediaType.APPLICATION_JSON))

.andExpect(status().isOk());

actions.andExpect(content().contentType(MediaType.APPLICATION_JSON)).andExpect(status().isOk())

.andExpect(jsonPath("$.vetList[0].id").value(1));

}

Spring MVC @Controller | AI Written Integration Tests

Obviously the developer spent some time writing these @Controller tests, let’s see how we could generate these test via the DiffBlue plugin and save some time! I have already gone through how to install the plugin in my last post so please refer to it if you don’t know how.

There is a plugin configuration that is needed to actually generate the Spring Integration tests. It’s just a check box that needs to be checked off. You can even specify which @Profile names to write tests for – very cool. I have left the Spring active profiles text box empty which means the default “all” Spring @Profile is used.

DiffBlue IntelliJ Plugin configuration for Spring Integration tests

All we have to do is right click on the VetController Class and select “Write Tests“. DiffBlue will analyze the source and byte code using its machine learning algorithms. At the end of the process, the following VetControllerDiffBlueTest was auto generated …

package org.springframework.samples.petclinic.vet;

import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;

import java.util.ArrayList;

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.mock.mockito.MockBean;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit.jupiter.SpringExtension;
import org.springframework.test.web.servlet.request.MockHttpServletRequestBuilder;
import org.springframework.test.web.servlet.request.MockMvcRequestBuilders;
import org.springframework.test.web.servlet.result.MockMvcResultMatchers;
import org.springframework.test.web.servlet.setup.MockMvcBuilders;

@ContextConfiguration(classes = {VetController.class})
@ExtendWith(SpringExtension.class)
public class VetControllerDiffBlueTest {
	@Autowired
	private VetController vetController;

	@MockBean
	private VetRepository vetRepository;

	@Test
	public void testConstructor() {
		// TODO: This test is incomplete.
		//   Reason: Nothing to assert: the constructed class does not have observers (e.g. getters or public fields).
		//   Add observers (e.g. getters or public fields) to the class.
		//   See https://diff.blue/R002

		new VetController(mock(VetRepository.class));
	}

	@Test
	public void testShowResourcesVetList() throws Exception {
		when(this.vetRepository.findAll()).thenReturn(new ArrayList<Vet>());
		MockHttpServletRequestBuilder requestBuilder = MockMvcRequestBuilders.get("/vets");
		MockMvcBuilders.standaloneSetup(this.vetController)
			.build()
			.perform(requestBuilder)
			.andExpect(MockMvcResultMatchers.status().isOk())
			.andExpect(MockMvcResultMatchers.content().contentType("application/xml"))
			.andExpect(MockMvcResultMatchers.content()
				.string("<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?><vets/>"));
	}

	@Test
	public void testShowVetList() throws Exception {
		when(this.vetRepository.findAll()).thenReturn(new ArrayList<Vet>());
		MockHttpServletRequestBuilder requestBuilder = MockMvcRequestBuilders.get("/vets.html");
		MockMvcBuilders.standaloneSetup(this.vetController)
			.build()
			.perform(requestBuilder)
			.andExpect(MockMvcResultMatchers.status().isOk())
			.andExpect(MockMvcResultMatchers.model().size(1))
			.andExpect(MockMvcResultMatchers.model().attributeExists("vets"))
			.andExpect(MockMvcResultMatchers.view().name("vets/vetList"))
			.andExpect(MockMvcResultMatchers.forwardedUrl("vets/vetList"));
	}

	@Test
	public void testShowVetList2() throws Exception {
		when(this.vetRepository.findAll()).thenReturn(new ArrayList<Vet>());
		MockHttpServletRequestBuilder getResult = MockMvcRequestBuilders.get("/vets.html");
		getResult.contentType("Not all who wander are lost");
		MockMvcBuilders.standaloneSetup(this.vetController)
			.build()
			.perform(getResult)
			.andExpect(MockMvcResultMatchers.status().isOk())
			.andExpect(MockMvcResultMatchers.model().size(1))
			.andExpect(MockMvcResultMatchers.model().attributeExists("vets"))
			.andExpect(MockMvcResultMatchers.view().name("vets/vetList"))
			.andExpect(MockMvcResultMatchers.forwardedUrl("vets/vetList"));
	}
}

package org.springframework.samples.petclinic.vet;

import static org.mockito.Mockito.mock;

import static org.mockito.Mockito.when;

import java.util.ArrayList;

import org.junit.jupiter.api.Test;

import org.junit.jupiter.api.extension.ExtendWith;

import org.springframework.beans.factory.annotation.Autowired;

import org.springframework.boot.test.mock.mockito.MockBean;

import org.springframework.test.context.ContextConfiguration;

import org.springframework.test.context.junit.jupiter.SpringExtension;

import org.springframework.test.web.servlet.request.MockHttpServletRequestBuilder;

import org.springframework.test.web.servlet.request.MockMvcRequestBuilders;

import org.springframework.test.web.servlet.result.MockMvcResultMatchers;

import org.springframework.test.web.servlet.setup.MockMvcBuilders;

@ContextConfiguration(classes = {VetController.class})

@ExtendWith(SpringExtension.class)

public class VetControllerDiffBlueTest {

@Autowired

private VetController vetController;

@MockBean

private VetRepository vetRepository;

@Test

public void testConstructor() {

// TODO: This test is incomplete.

// Reason: Nothing to assert: the constructed class does not have observers (e.g. getters or public fields).

// Add observers (e.g. getters or public fields) to the class.

// See https://diff.blue/R002

new VetController(mock(VetRepository.class));

}

@Test

public void testShowResourcesVetList() throws Exception {

when(this.vetRepository.findAll()).thenReturn(new ArrayList<Vet>());

MockHttpServletRequestBuilder requestBuilder = MockMvcRequestBuilders.get("/vets");

MockMvcBuilders.standaloneSetup(this.vetController)

.build()

.perform(requestBuilder)

.andExpect(MockMvcResultMatchers.status().isOk())

.andExpect(MockMvcResultMatchers.content().contentType("application/xml"))

.andExpect(MockMvcResultMatchers.content()

.string("<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?><vets/>"));

}

@Test

public void testShowVetList() throws Exception {

when(this.vetRepository.findAll()).thenReturn(new ArrayList<Vet>());

MockHttpServletRequestBuilder requestBuilder = MockMvcRequestBuilders.get("/vets.html");

MockMvcBuilders.standaloneSetup(this.vetController)

.build()

.perform(requestBuilder)

.andExpect(MockMvcResultMatchers.status().isOk())

.andExpect(MockMvcResultMatchers.model().size(1))

.andExpect(MockMvcResultMatchers.model().attributeExists("vets"))

.andExpect(MockMvcResultMatchers.view().name("vets/vetList"))

.andExpect(MockMvcResultMatchers.forwardedUrl("vets/vetList"));

}

@Test

public void testShowVetList2() throws Exception {

when(this.vetRepository.findAll()).thenReturn(new ArrayList<Vet>());

MockHttpServletRequestBuilder getResult = MockMvcRequestBuilders.get("/vets.html");

getResult.contentType("Not all who wander are lost");

MockMvcBuilders.standaloneSetup(this.vetController)

.build()

.perform(getResult)

.andExpect(MockMvcResultMatchers.status().isOk())

.andExpect(MockMvcResultMatchers.model().size(1))

.andExpect(MockMvcResultMatchers.model().attributeExists("vets"))

.andExpect(MockMvcResultMatchers.view().name("vets/vetList"))

.andExpect(MockMvcResultMatchers.forwardedUrl("vets/vetList"));

}

Comparing The Tests

The AI generated integration tests covered the same ground as the developer written tests however, it tested some things that the developer test did not. Surprisingly, these ended up being important.

Let me just start off by stating that the AI generated tests are very readable. The AI tests tend to use builder classes instead of using some annotations and don’t leverage “static imports”. This makes them slightly more verbose on the surface.

What I liked from the Developer Integration tests …

Uses @WebMvcTest which loads a slice of the Spring ApplicationContext in order to run faster.
The @Test testShowResourcesVetList method exercises the rendering of /vets endpoint URI as a JSON representation instead of the actual XML implementation. This test passes because the Jackon/JSON API classes are on the ClassPatth (via starter dependencies)

What I thought needed some improvement from the Developer Integration tests …

The testShowResourcesVetList relies on the Jackon/JSON library classes being on the ClassPath to leverage Spring’s MessageConverter service. This test could break in the future if the PetClinic Model Classes get more complicated for the Jackson library to infer since defaults are being used. Although a fine test, I would of first liked to see a @Test for “application/xml” as that is what is coded in the application!

What I liked from the DiffBlue AI generated Integration tests …

Covered the same ground that the developer tests did in terms of functionality – both endpoints are exercised. Huge time saver!
The testShowVetList went further in asserting extra expected state which had added value. It asserted that the model size must be equal to 1 (line 59) and also ensured that the view forwardedUrl was rendered (line 62). The developer test did not assert this.
The testShowResourcesVetList asserted that the HTTP response’s ContentType was “application/xml”(line 45) and that its actual source code content was in line with that (line 27). This tested the application’s actual implementation … Big uppercut!
An extra test testShowVetList2 was written for the /vets.html URI endpoint which sets the contentType to an unsupported value. This test actually comes back to haunt us when we later cause a regression!

What I thought needed some improvements from the DiffBlue AI generated Integration tests …

Use @WebMvcTest instead of using the larger @ContextConfiguration Spring Integration test slice. It would make the test run faster and consume less resources.
Need to take advantage of @BeforeEach to lower duplicate code in @Test

Overall the results were surprising! The AI generated tests did a better job than the developer tests. Of course, I just zoomed in on a single @Controller class but the writing is on the wall.

Don’t forget, we still need to write tests! It’s just that we can now concentrate on the tests that we feel are of the highest value. AI still can’t figure all of those out by itself. The best thing to do is to combine both.

In the above case, I would keep the developer test that exercised the JSON rendering and definitely keep the AI tests for the reasons already explained above.

How to Leverage AI Generated Tests in your Application

The bid idea is to use the DiffBlue plugin to generate all your Java Unit and Spring Integration tests across your code base. Once you have your own tests and DiffBlue’s tests written then you can move forward with confidence. This applies even if your application has zero tests! This plugin is worth gold in those cases.

Once you make a code change, you will need to re-generate new DiffBlue Spring Integration tests in order to have those new tests support/test your new behavior. The previous DiffBlue tests remain in order to assert that your past behavior (before change) does not change. If the previous regression tests do fail when run, then you have found a regression! Better now then in production.

You can see a walk-through of this in the YouTube tutorial where I actually catch a regression after making a breaking code change! The best thing to do is to automatically have the generation of the DiffBlue tests trigger in your CI/CD pipelines when you git commit or process a pull request.

AI Generated Tests Summary

Get use to it, these types of technologies and plugins are not going away. You have to start at least learning about how they can improve the quality of your product. Is it perfect? of course not but it has real value. We just saw how the AI generated tests were at the very least, just as good as the developer tests. In fact, they actually bested the developer tests. In the end, it’s best to combine both (if possible).

You don’t have to accept all tests produced by the plugin. I would go in there and even remove the tests that you identify may be redundant or add no value (i.e: the constructor @Test on line 29). Your still in control but it’s just that you now have so much more time to concentrate on what matters. We can’t ignore the cost savings as well, we’re talking big money savings … time is money!

Regressions are frustrating and embarrassing, especially when they occur in front of your clients. This is not the only plugin that tries to fill this space but its the best one out there. I’ll be using this as a new tool in my tool box for sure.

Share this post:

Twitter Facebook Reddit Email