Behaviour driven development
Behaviour driven development is different to test driven development. The 2 approaches are not necessarily mutually exclusive and are often used together.
The primary goal of behaviour driven development is to solve the problem of communication between the business (including the product manager), the engineering team and the machines.
If you think about a recent feature that’s been pushed to production, by the time a feature is deployed to production it will have moved through numerous stages, each with their own filters and interpretations. This set of chinese whispers is known as the cost of translation. The aim of behaviour driven development is to reduce the cost of translation.
Broadly speaking, BDD is meant to eliminate many of the issues that TDD introduces. BDD tests use a more verbose language so that they can be read almost like you would read a sentence. The ability to read tests like a sentence is a cognitive shift in how you think about your tests.
The argument is that if your engineers can read their tests fluidly, they will write better and more comprehensive tests. It’s often said that BDD is to help design the software, not test it and that TDD is meant to test it. Either way, BDD principles can help you and your team shift your mindset towards the behaviour of your product so that testing isn’t from a purely technical perspective.
Principles of BDD
BDD encourages simple languages to be used across teams, known as ubiquitous languages.
The simple and easy to use language should be used in the way the tests themselves are written, so that in theory, a business person can read a test and understand what it is testing.
Tests are often written from the customer’s point of view; the focus is on the customers and the users who are interacting with the product
Benefits of BDD
Simple language – the straightforward language is usable/understandable, not only by domain experts, but also by every member of the team.
Focus – BDD helps teams focus on a product’s behavioral elements rather than focusing on testing the technical implementation in isolation through individual units. This subtle, but important shift, means that everyone is focused on what the behaviour of the product should be.
Using Scenarios – BDD is designed to speed up the development process. Everyone involved in development relies upon the same scenarios. Scenarios are requirements, acceptance criteria, test cases, and test scripts all in one; there is no need to write any other artifact.
Efficiency – BDD frameworks make it easy to turn scenarios into automated tests. The steps are already given by the scenarios – the automation engineer simply needs to write a method/function to perform each step’s operations.
How to write BDD scenarios
Your first question is probably ‘what are BDD scenarios?’. That’s a fair question!
A BDD scenario is a written description of your product’s behavior from one or more users’ perspectives. Scenarios are designed to reduce the cost of translation and make it easier for your engineers to understand the requirements and for your QA (if you have one) to test it properly.
Using BDD scenarios means requirements and tests can be combined into 1 specificationx. In some cases, the scenarios that are written can then be easily converted into automated tests. BDD scenarios tend to follow a specific format. The format is fairly straight forward, and with a little practice you’ll be able to write your own.
In order to explain how BDD and scenarios work in practice, let’s take a look at the example of a user signing up to LinkedIn.
Example – signing up for a LinkedIn account
Here’s a basic BDD scenario which describes the LinkedIn signup process:
Scenario 1: User successfully creates a LinkedIn Account
GIVEN John is on the LinkedIn Registration page
WHEN he enters all required registration fields
THEN a LinkedIn account is created
Let’s take a look at what’s going on here.
The first thing you’ll notice is the header ‘Scenario 1: user successfully creates a LinkedIn Account’. All written BDD scenarios should be given a header which accurately describes the scenario you’re interested in. You may have a few scenarios to assist your engineers / QA team and without headers things can get messy.
The second thing you’ll notice is the use of 3 words: GIVEN, WHEN, THEN. They don’t necessarily need to be in capital letters but, in a similar way to SQL, sometimes keywords are made more clear if they’re in caps.
GIVEN, WHEN, THEN
In its simplest format, there are 3 key elements in any BDD scenario:
GIVEN (describing the context) WHEN (describing the action) THEN (describing the outcome)
These 3 elements help to describe the behaviour of the system using context, actions and outcomes. If you have more than one or you require more information than this, you can add them with AND.
Using ANDs
When you require more information in a scenario, an ‘AND’ can be used after any of the descriptors:
GIVEN (context), AND (further context), WHEN (action/event), AND (further action/event), THEN (outcome) AND (further outcome)
Make sense?
OK, let’s revisit our LinkedIn registration scenario and flesh it out in more detail with a few ANDs.
GIVEN John is on LinkedIn Registration page WHEN he enters all the required registration information AND he hits ‘join now’ THEN his LinkedIn account is created AND he is directed to the profile creation page AND his confirmation email is sent
How do BDD scenarios work with user stories?
Your BDD scenarios should form part of 1 specification. This includes both the user story and the scenarios. So, for example, if you’re writing a user story in Jira or some other beloved backlog management tool you could structure your specification in Jira in the following order:
User story – start with the user story describing the requirements from a user’s perspective
BDD scenarios – then include your scenarios describing the behaviour of the system to assist with testing
A user story has a slightly different structure to your BDD. Your user story should comprise the following:
As an X I can / want Y So that Z
Using our LinkedIn example:
As a new user (John), I can register a new account on the homepage so that I can access LinkedIn.
Your BDD scenarios would then follow:
Scenario 1: User successfully creates a LinkedIn Account
GIVEN John is on LinkedIn Registration page WHEN he enters all the required registration information AND he hits ‘join now’ THEN his LinkedIn account is created AND he is directed to the profile creation page AND his confirmation email is sent
If you have multiple scenarios, you’d add these after Scenario 1 in a sequence.
When should BDD scenarios be used?
BDD scenarios are not necessarily mandatory across all of your product specifications. If they were you’d spend most of your life writing BDD scenarios which would clearly be very unpleasant.
BDD scenarios are best suited to specifications where you think there is a likelihood that the requirements may be misunderstood without them or that a more thorough testing approach needs to be adopted. If you’re working on a small color change, text change or a technical chore / bug, there will clearly be no case for using BDD scenarios as this would be a waste of everyone’s time.
It may be that you as a team decide to write them for all major new feature stories or that you only focus on a specific type of specification. BDD scenarios can assist you in your development process but as with all things product, you and your team should decide on what works best for you.
How am I supposed to write my Gherkin steps?
Good Gherkin feature files are not easy to write at first. Writing is definitely an art. With some basic pointers, and a bit of practice, Gherkin becomes easier.
The Golden Gherkin Rule: Treat other readers as you would want to be treated. Write Gherkin so that people who don’t know the feature will understand it.
Proper Behavior
The biggest mistake BDD beginners make is writing Gherkin without a behavior-driven mindset. They often write feature files as if they are writing “traditional” procedure-driven functional tests: step-by-step instructions with actions and expected results. HP ALM, qTest, and many other test repository tools store tests in this format. These procedure-driven tests are often imperative and trace a path through the system that covers multiple behaviors. As a result, they may be unnecessarily long, which can delay failure investigation, increase maintenance costs, and create confusion.
For example, let’s consider a test that searches for images of pandas on Google. Below would be a reasonable test procedure:
Open a web browser.
Web browser opens successfully.
Navigate to https://www.google.com/.
The web page loads successfully and the Google image is visible.
Enter “panda” in the search bar.
Links related to “panda” are shown on the results page.
Click on the “Images” link at the top of the results page.
Images related to “panda” are shown on the results page.
I’ve seen many newbies translate a test like this into Gherkin like the following:
# BAD EXAMPLE! Do not copy.
Feature: Google Searching
Scenario: Google Image search shows pictures
Given the user opens a web browser
And the user navigates to "https://www.google.com/"
When the user enters "panda" into the search bar
Then links related to "panda" are shown on the results page
When the user clicks on the "Images" link at the top of the results page
Then images related to "panda" are shown on the results page
This scenario is terribly wrong. All that happened was that the author put BDD buzzwords in front of each step of the traditional test. This is not behavior-driven, it is still procedure-driven.
The first two steps are purely setup: they just go to Google, and they are strongly imperative. Since they don’t focus on the desired behavior, they can be reduced to one declarative step: “Given a web browser is at the Google home page.” This new step is friendlier to read.
After the Given step, there are two When-Then pairs. This is syntactically incorrect: Given-When-Then steps must appear in order and cannot repeat. A Given may not follow a When or Then, and a When may not follow a Then. The reason is simple: any single When-Then pair denotes an individual behavior. This makes it easy to see how, in the test above, there are actually two behaviors covered: (1) searching from the search bar, and (2) performing an image search. In Gherkin, one scenario covers one behavior. Thus, there should be two scenarios instead of one. Any time you want to write more than one When-Then pair, write separate scenarios instead. (Note: Some BDD frameworks may allow disordered steps, but it would nevertheless be anti-behavioral.)
This splitting technique also reveals unnecessary behavior coverage. For instance, the first behavior to search from the search bar may be covered in another feature file. I once saw a scenario with about 30 When-Then pairs, and many were duplicate behaviors.
Do not be tempted to arbitrarily reassign step types to make scenarios follow strict Given-When-Then ordering. Respect the integrity of the step types: Givens set up initial state, Whens perform an action, and Thens verify outcomes. In the example above, the first Then step could have been turned into a When step, but that would be incorrect because it makes an assertion. Step types are meant to be guide rails for writing good behavior scenarios.
The correct feature file would look something like this:
Feature: Google Searching
Scenario: Search from the search bar
Given a web browser is at the Google home page
When the user enters "panda" into the search bar
Then links related to "panda" are shown on the results page
Scenario: Image search
Given Google search results for "panda" are shown
When the user clicks on the "Images" link at the top of the results page
Then images related to "panda" are shown on the results page
The second behavior arguably needs the first behavior to run first because the second needs to start at the search result page. However, since that is merely setup for the behavior of image searching and is not part of it, the Given step in the second scenario can basically declare (declaratively) that the “panda” search must already be done. Of course, this means that the “panda” search would be run redundantly at test time, but the separation of scenarios guarantees behavior-level independence.
The Cardinal Rule of BDD: One Scenario, One Behavior!
Remember, behavior scenarios are more than tests – they also represent requirements and acceptance criteria. Good Gherkin comes from good behavior.
Phrasing Steps
How you write a step matters. If you write a step poorly, it cannot easily be reused. Thankfully, some basic rules maintain consistent phrasing and maximum reusability.
Write all steps in third-person point of view. If first-person and third-person steps mix, scenarios become confusing.
Write steps as a subject-predicate action phrase. It may tempting to leave parts of speech out of a step line for brevity, especially when using Ands and Buts, but partial phrases make steps ambiguous and more likely to be reused improperly. For example, consider the following example:
# BAD EXAMPLE! Do not copy.
Feature: Google Searching
Scenario: Google search result page elements
Given the user navigates to the Google home page
When the user entered "panda" at the search bar
Then the results page shows links related to "panda"
And image links for "panda"
And video links for "panda"
The final two And steps lack the subject-predicate phrase format. Are the links meant to be subjects, meaning that they perform some action? Or, are they meant to be direct objects, meaning that they receive some action? Are they meant to be on the results page or not? What if someone else wrote a scenario for a different page that also had image and video links – could they reuse these steps? Writing steps without a clear subject and predicate is not only poor English but poor communication.
Also, use appropriate tense for each type of step. Givens should always use present or present perfect tense, and Whens and Thens should always use present tense. Rather than take a time warp back to middle school English class, let’s illustrate tense with a bad example:
# BAD EXAMPLE! Do not copy.
Feature: Google Searching
Scenario: Simple Google search
Given the user navigates to the Google home page
When the user entered "panda" at the search bar
Then links related to "panda" will be shown on the results page
The Given step above indicates an action when it says, “The user navigates.” Actions imply the exercise of behavior. However, Given steps are meant to establish an initial state, not exercise a behavior. This may seem like a trivial nuance, but it can confuse feature file authors who may not be able to tell if a step is a Given or When. Using present or present perfect tense indicates a state rather than an action.
The When step above uses past tense when it says, “The user entered.” This indicates that an action has already happened. However, When steps should indicate that an action is presently happening. Plus, past tense here conflicts with the tenses used in the other steps.
The Then step above uses future tense when it says, “The results will be shown.” Future tense seems practical for Then steps because it indicates what the result should be after the current action is taken. However, future tense reinforces a procedure-driven approach because it treats the scenario as a time sequence. A behavior, on the other hand, is a present-tense aspect of the product or feature. Thus, it is better to write Then steps in the present tense.
The corrected example looks like this:
Feature: Google Searching
Scenario: Simple Google search
Given a web browser is at the Google home page
When the user enters "panda" into the search bar
Then links related to "panda" are shown on the results page
And note, all steps are written in third-person.
Good Titles
Good titles are just as important as good steps. The title is like the face of a scenario – it’s the first thing people read. It must communicate in one concise line what the behavior is. Titles are often logged by the automation framework as well.
Choices, Choices
Another common misconception for beginners is thinking that Gherkin has an “Or” step for conditional or combinatorial logic. People may presume that Gherkin has “Or” because it has “And”, or perhaps programmers want to treat Gherkin like a structured language. However, Gherkin does not have an “Or” step. When automated, every step is executed sequentially.
Below is a bad example based on a classic Super Mario video game, showing how people might want to use “Or”:
# BAD EXAMPLE! Do not copy.
Feature: SNES Mario Controls
Scenario: Mario jumps
Given a level is started
When the player pushes the "A" button
Or the player pushes the "B" button
Then Mario jumps straight up
Clearly, the author’s intent is to say that Mario should jump when the player pushes either of two buttons. The author wants to cover multiple variations of the same behavior. In order to do this the right way, use Scenario Outline sections to cover multiple variations of the same behavior, as shown below:
Feature: SNES Mario Controls
Scenario Outline: Mario jumps
Given a level is started
When the player pushes the "<letter>" button
Then Mario jumps straight up
Examples: Buttons
| letter |
| A |
| B |
The Known Unknowns
Test data can be difficult to handle. Sometimes, it may be possible to seed data in the system and write tests to reference it, but other times, it may not. Google search is the prime example: the result list will change over time as both Google and the Internet change. To handle the known unknowns, write scenarios defensively so that changes in the underlying data do not cause test runs to fail. Furthermore, to be truly behavior-driven, think about data not as test data but as examples of behavior.
Consider the following example from the previous :
Feature: Google Searching
Scenario: Simple Google search
Given a web browser is on the Google page
When the search phrase "panda" is entered
Then results for "panda" are shown
And the following related results are shown
| related |
| Panda Express |
| giant panda |
| panda videos |
This scenario uses a step table to explicitly name results that should appear for a search. The step with the table would be implemented to iterate over the table entries and verify each appeared in the result list. However, what if Panda Express were to go out of business and thus no longer be ranked as high in the results? (Let’s hope not.) The test run would then fail, not because the search feature is broken, but because a hard-coded variation became invalid. It would be better to write a step that more intelligently verified that each returned result somehow related to the search phrase, like this: “And links related to ‘panda’ are shown on the results page.” The step definition implementation could use regular expression parsing to verify the presence of “panda” in each result link.
Another nice feature of Gherkin is that step definitions can hide data in the automation when it doesn’t need to be exposed. Step definitions may also pass data to future steps in the automation. For example, consider another Google search scenario:
Feature: Google Searching
Scenario: Search result linking
Given Google search results for "panda" are shown
When the user clicks the first result link
Then the page for the chosen result link is displayed
Notice how the When step does not explicitly name the value of the result link – it simply says to click the first one. The value of the first link may change over time, but there will always be a first link. The Then step must know something about the chosen link in order to successfully verify the outcome, but it can simply reference it as “the chosen result link”. Behind the scenes, in the step definitions, the When step can store the value of the chosen link in a variable and pass the variable forward to the Then step.
Handling Test Data
Some types of test data should be handled directly within the Gherkin, but other types should not. Remember that BDD is specification by example – scenarios should be descriptive of the behaviors they cover, and any data written into the Gherkin should support that descriptive nature.
Less is More
Scenarios should be short and sweet. I typically recommend that scenarios should have a single-digit step count (<10). Long scenarios are hard to understand, and they are often indicative of poor practices. One such problem is writing imperative steps instead of declarative steps. I have touched on this topic before, but I want to thoroughly explain it here.
Imperative steps state the mechanics of how an action should happen. They are very procedure-driven. For example, consider the following When steps for entering a Google search:
When the user scrolls the mouse to the search bar
And the user clicks the search bar
And the user types the letter “p”
And the user types the letter “a”
And the user types the letter “n”
And the user types the letter “d”
And the user types the letter “a”
And the user types the ENTER key
Now, the granularity of actions may seem like overkill, but it illustrates the point that imperative steps focus very much on how actions are taken. Thus, they often need many steps to fully accomplish the intended behavior. Furthermore, the intended behavior is not always as self-documented as with declarative steps.
Declarative steps state what action should happen without providing all of the information for how it will happen. They are behavior-driven because they express action at a higher level. All of the imperative steps in the example above could be written in one line: “When the user enters ‘panda’ at the search bar.” The scrolling and keystroking is implied, and it will ultimately be handled by the automation in the step definition. When trying to reduce step count, ask yourself if your steps can be written more declaratively.
Another reason for lengthy scenarios is scenario outline abuse. Scenario outlines make it all too easy to add unnecessary rows and columns to their Examples tables. Unnecessary rows waste test execution time. Extra columns indicate complexity. Both should be avoided. Below are questions to ask yourself when facing an oversized scenario outline:
Does each row represent an equivalence class of variations?
For example, searching for “elephant” in addition to “panda” does not add much test value.
Does every combination of inputs need to be covered?
N columns with M inputs each generates MN possible combinations.
Consider making each input appear only once, regardless of combination.
Do any columns represent separate behaviors?
This may be true if columns are never referenced together in the same step.
If so, consider splitting apart the scenario outline by column.
Does the feature file reader need to explicitly know all of the data?
Consider hiding some of the data in step definitions.
Some data may be derivable from other data.
These questions are meant to be sanity checks, not hard-and-fast rules. The main point is that scenario outlines should focus on one behavior and use only the necessary variations.
Style and Structure
While style often takes a backseat during code review, it is a factor that differentiates good feature files from great feature files. In a truly behavior-driven team, non-technical stakeholders will rely upon feature files just as much as the engineers. Good writing style improves communication, and good communication skills are more than just resume fluff.
Below are a number of tidbits for good style and structure:
Focus a feature on customer needs.
Limit one feature per feature file. This makes it easy to find features.
Limit the number of scenarios per feature. Nobody wants a thousand-line feature file. A good measure is a dozen scenarios per feature.
Limit the number of steps per scenario to less than ten.
Limit the character length of each step. Common limits are 80-120 characters.
Use proper spelling.
Use proper grammar.
Capitalize Gherkin keywords.
Capitalize the first word in titles.
Do not capitalize words in the step phrases unless they are proper nouns.
Do not use punctuation (specifically periods and commas) at the end of step phrases.
Use single spaces between words.
Indent the content beneath every section header.
Separate features and scenarios by two blank lines.
Separate examples tables by 1 blank line.
Do not separate steps within a scenario by blank lines.
Space table delimiter pipes (“|”) evenly.
Adopt a standard set of tag names. Avoid duplicates.
Write all tag names in lowercase, and use hyphens (“-“) to separate words.
Limit the length of tag names.
Without these rules, you might end up with something like this:
# BAD EXAMPLE! Do not copy.
Feature: Google Searching
@AUTOMATE @Automated @automation @Sprint32GoogleSearchFeature
Scenario outline: GOOGLE STUFF
Given a Web Browser is on the Google page,
when The seach phrase "<phrase>" Enter,
Then "<phrase>" shown.
and The relatedd results include "<related>".
Examples: animals
| phrase | related |
| panda | Panda Express |
| elephant | elephant Man |
Don’t do this. It looks horrible. While the automation code may look hairy in parts, Gherkin files should look elegant.
Comments