Using scenarios as test cases


Anti-patterns in BDD

Using BDD can be a great way to increase collaboration and conversation between team members. It can help teams with their shared understanding and keep teams on track with their delivery. However, succeeding with BDD requires patience, teamwork and feedback - both on what works and what doesn’t work for your team.

Working as a team can be fraught with BDD anti-patterns, whether they’re caused by biases, losing sight of common goals, or relying too heavily on tooling. In this series, we are looking at some of these anti-patterns, uncovering what they look like, how they are formed, and how they can be avoided.

In today's article, we'll have a closer look into the anti-pattern of using scenarios as test cases.

Using scenarios as test cases

Imagine a team that is working on a product in which the testers have decided to adopt what they claim to be a ‘BDD’ approach. Their version of ‘BDD’ involves:

  • The test team creating a series of test cases written using Gherkin syntax.

  • Each test case is written to test and confirm a specific acceptance criteria or expectation.

  • Developing the product based on the test cases.

  • The test team running through these test cases to test new features, and for regression testing purposes.

Essentially, the testers’ ‘BDD’ approach is happening back to front.

Why is using scenarios as test cases an anti-pattern?

Writing test cases in Gherkin has two key problems which relate firstly from the perspective of a test case, and secondly from the perspective of Gherkin.

From the perspective of a test case, Gherkin syntax is a poor alternative. A useful test case will contain steps to carry out, expected outcomes, areas to add in observations, and other supporting data for your testing. All of which can be used in a wide range of testing activities both high-level user-focused and also low-level technical-focused.

Gherkin is designed to describe behaviour on a high, or abstract, level. Writing test scenarios in Gherkin creates a bias in that it forces you to write them in a high-level manner. Whilst it’s important for the agile team to be able to consider behaviour at a high level. For the product to be successful, testing is required at other interfaces and layers. Additionally, with test cases we can detail actions, expected outcomes and test data, along with other comprehensive detail that can be valuable to a test case. Attempting to cram that content into the Gherkin format makes the scenarios hard to read, lacking in clarity and unhelpful.

From the perspective of Gherkin, we use scenarios and examples to describe the valuable behaviour of products, which is different to exhaustively describing every minute detail of the product. Creating hundreds, or thousands, of test cases in Gherkin format prevents that detailed description from being included, and detracts from the value that well-formed examples give.

When we use test cases, we are using them as a means to validate expectations. In BDD we use examples, or scenarios, as a means to share understanding. Gherkin is simply a way of capturing those examples in a standardised way which, as an artefact, may or may not be used later in the development process for outside-in development automation, and/or living documentation. So attempting to use Gherkin for test cases we lose the value of both the detail of test cases and the output of shared conversations.

What’s the solution?

The first step towards fixing this anti-pattern is to separate your testing and test cases from your Gherkin examples, which have been created for separate purposes. Next, if you aren’t already doing so, start collaborating more as a team before development begins. By having conversations you will gain shared understanding, and you will see benefits.

Finally, if you want to use scenarios, start by sharing examples in your collaborative sessions to describe requirements and behaviour. For example:

  • For the behaviour of a travel app when a train is delayed

“When the 10:15 from London Liverpool Street is delayed by 10 minutes the system sends a text message saying the ‘The London Kings Cross train is delayed by 10 minutes’”

  • For the behaviour of a calendar app when saving events

“When I plan to go round to my friend Alan’s house on Tuesday 10th April at 7pm, I can choose 10th April in my calendar and save an event saying ‘Dinner at Alan’s’ and a time of 7pm”

Examples like these will help to clearly illustrate what the business representative or user wants. Then, if you want to use those examples as artefacts for other activities, that’s when you can explore how to capture your examples in a Gherkin syntax.

There's more where that came from...

This is blog post 2 of our 4 part anti-pattern series. You can find links to these below:

Anti-pattern post 1 - Missing an amigo

Anti-pattern post 3 - Documenting acceptance criteria as individual scenarios

Anti-pattern post 4 - Automated scenarios is enough test coverage

About the author

Mark Winteringham, aka @2bittester , is a coach, teacher, mentor, tester and international speaker, and ranked joint 8th in Agile Testing Days 2017 list of most influential agile testing professionals. Mark has worked on award-winning projects using various web, mobile and desktop technologies and he has been an agent for change for clients across a wide range of sectors, implementing successful BDD and automation in testing strategies. Mark’s focus for the past year or so has been educating teams on how not to use BDD, and he co-developed and delivers the Hindsight 2-day BDD & Cucumber training course.

You may also like…