Smart BDD vs. Cucumber Using Java and JUnit5

Cucumber is the leading Behavior-driven development (BDD) framework. It is language-agnostic and integrates with other frameworks. You write the specification/feature, then write the glue code, then write the test code.

With Smart BDD, you write the code first using best practices, and this generates the following:

• Interactive feature files that serve as documentation
• Diagrams to better document the product

The barrier to entry is super low. You start with one annotation or add a file to resources/META-INF!

That's it. You're generating specification/documentation. Please note I will interchange specifications, features, and documentation throughout.

If you haven't seen Smart BDD before, here's an example:

The difference in approach leads to Smart BDD:

• To have less code and higher quality code
• Therefore, less complexity
• Therefore, lowering the cost of maintaining and adding testing
• Therefore, increasing productivity
• Oh, and you get sequence diagrams (see picture above), plus many new features are in the pipeline

Both goals are the same, in a nutshell — specifications that can be read by anyone and tests that are exercised.

Implementing BDD with Cucumber will give you benefits. However, there is a technical cost to adding and maintaining feature files. This means extra work has to be done.

There are three main layers: feature file, glue code, and test code:

• You write the feature file
• Then the glue code
• Then the test code

This approach, with extra layers and workarounds for limitations and quirks, leads Cucumber (we'll explore in more with code detail below):

• To have more code and lower quality. You have to work around limitations and quirks.
• Therefore, more complexity
• Therefore, increasing the cost of maintaining and adding testing
• Therefore, decreasing productivity
• Therefore, decreased coverage

The quality of code can be measured in its ability to change! Hence, best practices and less code fulfill this brief.

It's time to try and back these claims up. Let's check out the latest examples from Cucumber.

For example, below, I created a repo for one small example — calculator-java-junit5. Then, I copied and pasted it into a new project.

First, Let’s Implement the Cucumber Solution

Feature file:

Feature: Shopping

  Scenario: Give correct change
    Given the following groceries:
      | name  | price |
      | milk  | 9     |
      | bread | 7     |
      | soap  | 5     |
    When I pay 25
    Then my change should be 4

Java source code:

public class ShoppingSteps {

    private final RpnCalculator calc = new RpnCalculator();

    @Given("the following groceries:")
    public void the_following_groceries(List<Grocery> groceries) {
        for (Grocery grocery : groceries) {
            calc.push(grocery.price.value);
            calc.push("+");
        }
    }

    @When("I pay {}")
    public void i_pay(int amount) {
        calc.push(amount);
        calc.push("-");
    }

    @Then("my change should be {}")
    public void my_change_should_be_(int change) {
        assertEquals(-calc.value().intValue(), change);
    }
    // omitted Grocery and Price class
}

Mapping for test input:

public class ParameterTypes {

    private final ObjectMapper objectMapper = new ObjectMapper();

    @DefaultParameterTransformer
    @DefaultDataTableEntryTransformer
    @DefaultDataTableCellTransformer
    public Object transformer(Object fromValue, Type toValueType) {
        return objectMapper.convertValue(fromValue, objectMapper.constructType(toValueType));
    }
}

Test runner:

/**
 * Work around. Surefire does not use JUnits Test Engine discovery
 * functionality. Alternatively execute the
 * org.junit.platform.console.ConsoleLauncher with the maven-antrun-plugin.
 */
@Suite
@IncludeEngines("cucumber")
@SelectClasspathResource("io/cucumber/examples/calculator")
@ConfigurationParameter(key = GLUE_PROPERTY_NAME, value = "io.cucumber.examples.calculator")
public class RunCucumberTest {
}

build.gradle.kts showing the cucumber config:

dependencies {
    testImplementation("io.cucumber:cucumber-java")
    testImplementation("io.cucumber:cucumber-junit-platform-engine")
}

tasks.test {
    // Work around. Gradle does not include enough information to disambiguate
    // between different examples and scenarios.
    systemProperty("cucumber.junit-platform.naming-strategy", "long")
}

Secondly, We Will Implement the Smart BDD Solution

Java source code:

@ExtendWith(SmartReport.class)
public class ShoppingTest {
    private final RpnCalculator calculator = new RpnCalculator();

    @Test
    void giveCorrectChange() {
        givenTheFollowingGroceries(
            item("milk", 9),
            item("bread", 7),
            item("soap", 5));
        whenIPay(25);
        myChangeShouldBe(4);
    }

    public void whenIPay(int amount) {
        calculator.push(amount);
        calculator.push("-");
    }

    public void myChangeShouldBe(int change) {
        assertThat(-calculator.value().intValue()).isEqualTo(change);
    }

    public void givenTheFollowingGroceries(Grocery... groceries) {
        for (Grocery grocery : groceries) {
            calculator.push(grocery.getPrice());
            calculator.push("+");
        }
    }
    // omitted Grocery class 
}

build.gradle.kts showing the Smart BDD config:

dependencies {
    testImplementation("io.bit-smart.bdd:report:0.1-SNAPSHOT")
}

This generates:

Scenario: Give correct change (PASSED)
Given the following groceries
  "milk" 9
  "bread" 7
  "soap" 5
When I pay 25
My change should be 4

Notice how simple Smart BDD is, with much fewer moving parts — 1 test class vs 4 files.
We removed the Cucumber feature file. The feature file has a few main drawbacks:

• It adds the complexity of mapping between itself and the source code
• As an abstraction, it will leak into the bottom layers
• It is very hard to keep feature files consistent
• When developing an IDE, it will need to support the feature file. Frequently you'll be left with no support

You don't have these drawbacks in Smart BDD. In fact, it promotes best practices and productivity.

The counterargument for feature files is normally, well, it allows non-devs to create user stories and or acceptance criteria. The reality is when a product owner writes a user story and or acceptance criteria, it will almost certainly be modified by the developer. Using Smart BDD, you can still write user stories and or acceptance criteria in your backlog. It's a good starting point to help you write the code. In time you'll end up with more consistency.

In the Next Section, I’ll Try To Demonstrate the Complexity of Cucumber

Let's dive into something more advanced:

• A dollar is 2 of the currency below
• Visa payments take 1 currency processing fee

When I pay 25 "Dollars"
Then my change should be 29

It is reasonable to think that we can add this method:

@When("I pay {int} {string}")
public void i_pay(int amount,String currency){
    calc.push(amount*exchangeRate(currency));
    calc.push("-");
}

However, this is the output:

Step failed
io.cucumber.core.runner.AmbiguousStepDefinitionsException: "I pay 25 "Dollars"" matches more than one step definition:
"I pay {int} {string}" in io.cucumber.examples.calculator.ShoppingSteps.i_pay(int,java.lang.String)

Here is where the tail starts to wag the dog. You embark on investing time and more code to work around the framework. We should always strive for simplicity and additional code, and in a boarder sense, additional features will always make code harder to maintain.

We have three options:

1. Mutate i_pay method to handle a currency. If we had 10's or 100's, occurrences of When I pay .. this would be risky and time-consuming. If we add a "Visa" payment method, we are starting to add complexity to an existing method.
2. Create a new method that doesn't start with I pay. It could be With currency I pay 25 "Dollars". Not ideal, as this isn't really what I wanted. It loses discoverability. How would we add a "Visa" payment method?
3. Use multiple steps I pay and with currency. This is the most maintainable solution. For discoverability, you'd need a consistent naming convention. With a large codebase, good luck with discoverability, as they are loosely coupled in the feature file but coupled in code.

Option 1 is the one I have seen the most — God glues methods with very complicated regular expressions. With Cucumber Expressions, it's the cleanest code I have seen. According to the Cucumber documentation, conjunction steps are an anti-pattern. If I added a payment method I pay 25 "Dollars" with "Visa" I don't know if this constitutes the conjunction step anti-pattern. If we get another requirement, "Visa" payments doubled on a "Friday," setting the day surely constitutes another step.

Option 3 is really a thin layer on a builder. Below is one possible implementation of a builder. With this approach, adding the day of the week would be trivial (as we've chosen to use the builder pattern).

When I pay 25
And with currency "Dollars"

public class ShoppingSteps {

    private final ShoppingService shoppingService = new ShoppingService();
    private final PayBuilder payBuilder = new PayBuilder();

    @Given("the following groceries:")
    public void the_following_groceries(List<Grocery> groceries) {
        for (Grocery grocery : groceries) {
            shoppingService.calculatorPush(grocery.getPrice().getValue());
            shoppingService.calculatorPush("+");
        }
    }

    @When("I pay {int}")
    public void i_pay(int amount) {
        payBuilder.withAmount(amount);
    }

    @When("with currency {string}")
    public void i_pay_with_currency(String currency) {
        payBuilder.withCurrency(currency);
    }

    @Then("my change should be {}")
    public void my_change_should_be_(int change) {
        pay();
        assertThat(-shoppingService.calculatorValue().intValue()).isEqualTo(change);
    }

    private void pay() {
        final Pay pay = payBuilder.build();
        shoppingService.calculatorPushWithCurrency(pay.getAmount(), pay.getCurrency());
        shoppingService.calculatorPush("-");
    }
    // builders and classes omitted
}

Let’s Implement This in Smart BDD

@ExtendWith(SmartReport.class)
public class ShoppingTest {
    private final ShoppingService shoppingService = new ShoppingService();
    private PayBuilder payBuilder = new PayBuilder();

    @Test
    void giveCorrectChange() {
        givenTheFollowingGroceries(
            item("milk", 9),
            item("bread", 7),
            item("soap", 5));
        whenIPay(25);
        myChangeShouldBe(4);
    }

    @Test
    void giveCorrectChangeWhenCurrencyIsDollars() {
        givenTheFollowingGroceries(
            item("milk", 9),
            item("bread", 7),
            item("soap", 5));
        whenIPay(25).withCurrency("Dollars");
        myChangeShouldBe(29);
    }

    public PayBuilder whenIPay(int amount) {
        return payBuilder.withAmount(amount);
    }

    public void myChangeShouldBe(int change) {
        pay();
        assertEquals(-shoppingService.calculatorValue().intValue(), change);
    }

    public void givenTheFollowingGroceries(Grocery... groceries) {
        for (Grocery grocery : groceries) {
            shoppingService.calculatorPush(grocery.getPrice());
            shoppingService.calculatorPush("+");
        }
    }

    private void pay() {
        final Pay pay = payBuilder.build();
        shoppingService.calculatorPushWithCurrency(pay.getAmount(), pay.getCurrency());
        shoppingService.calculatorPush("-");
    }
    // builders and classes omitted
}

Let's count the number of lines for the solution of optionally paying with dollars:

• Cucumber: 
  • ShoppingSteps 123
  • ParameterTypes 21
  • RunCucumberTest 16 
  • shopping.feature 20 
  • Total: 180 lines
• Smart BDD: 
  • ShoppingTest 114 lines
  • Total: 114 lines

Hopefully, I have demonstrated the simplicity and productivity of Smart BDD.

Example of Using Diagrams With Smart BDD

This is the source code:

@ExtendWith(SmartReport.class)
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class BookControllerIT {
    // skipped setup...

   @Override
   public void doc() {
     featureNotes("Working progress for example of usage Smart BDD");
   }
   
   @BeforeEach
   void setupUml() {
       sequenceDiagram()
         .addActor("User")
         .addParticipant("BookStore")
         .addParticipant("ISBNdb");
   }
   
   @Order(0)
   @Test
   public void getBookBy13DigitIsbn_returnsTheCorrectBook() {
     whenGetBookByIsbnIsCalledWith(VALID_13_DIGIT_ISBN_FOR_BOOK_1);
     thenTheResponseIsEqualTo(BOOK_1);
   }

   private void whenGetBookByIsbnIsCalledWith(String isbn) {
      HttpHeaders headers = new HttpHeaders();
      headers.setAccept(singletonList(MediaType.APPLICATION_JSON));
      response = template.getForEntity("/book/" + isbn, String.class, headers);
      generateSequenceDiagram(isbn, response, headers);
   }

   private void generateSequenceDiagram(String isbn, ResponseEntity<String> response, HttpHeaders headers) {
      sequenceDiagram().add(aMessage().from("User").to("BookStore").text("/book/" + isbn));
      
      List<ServeEvent> allServeEvents = getAllServeEvents();
      allServeEvents.forEach(event -> {
         sequenceDiagram().add(aMessage().from("BookStore").to("ISBNdb").text(event.getRequest().getUrl()));
         sequenceDiagram().add(aMessage().from("ISBNdb").to("BookStore").text(
           event.getResponse().getBodyAsString() +  " [" + event.getResponse().getStatus() + "]"));
      });

      sequenceDiagram().add(aMessage().from("BookStore").to("User").text(response.getBody() + " [" + response.getStatusCode().value() + "]"));
   }

    // skipped helper classes...
}

In my opinion, the above does a very good job of documenting the Book Store.

Smart BDD is being actively developed. I'll try to reduce the code required for diagrams, and potentially use annotations. Strike a balance between magic and declarative code.

I use the method whenGetBookByIsbnIsCalledWith in the example above, as this is the most appropriate abstraction. If we had more requirements, then the code could look more like the one below. This is at the other end of the spectrum. Work has gone into a test API to make testing super easy. With its approach, notice how consistent the generated documentation will be. It will make referring to the documentation much easier.

public class GetBookTest extends BaseBookStoreTest {

    @Override
    public void doc() {
        featureNotes("Book Store example of usage Smart BDD");
    }

    @Test
    public void getBookWithTwoAuthors() {
        given(theIsbnDbContains(aBook().withAuthors("author", "another-author")));
        when(aUserRequestsABook());
        then(theResponseContains(aBook().withAuthors("author", "another-author")));
    }
}

SmartBDD allows me to choose the abstraction/solution that I feel is right without a framework getting in the way or adding to my workload.

Anything you do and don't like, please comment below. I encourage anybody to contact me if you want to know more — contact details on GitHub.

All source code can be found here.

Please check out GitHub and Smart BDD website