BlogAnatomy of a User Story: Acceptance Criteria

Where does acceptance criteria fit in to user stories?

Consider a user story as an outline of literary short story. The initial narrative (covered in the previous installment) reveals the WHO, WHAT, and WHY of a feature. It’s a summary, giving the reader context about one or more main characters who need to accomplish something so a problem can ultimately be solved. Now, the next section in literary outline is the acceptance criteria, painting a picture of what it looks like for the characters to succeed in their efforts. Since we are the authors, we determine the fate of the characters, and our characters must always succeed, even if they stumble at times.

What is DONE?

Acceptance criteria (AC) are essential to user stories, and have various consumers in the software team and among stakeholders. However if I could sum up AC in the smallest number of words, it would be: AC is the definition of done for a user story. At any point during implementation of a feature (in UX design, prototyping, and/or in development), anyone should be able to read the AC and come to a determination if the feature fulfills each of the criteria.

Before going too much further, let’s jump right in to learn from an example.

User story acceptance criteria example

The following AC for a user story of a software application enabling students to take online courses. While the example is contrived, you can certainly use this as a model for how to write acceptance criteria. So first the user story narrative is below for context, followed by the acceptance criteria:

As a student
I want to search through the course catalog for a course by keyword
So that I can easily find a course name or topic that is of interest to me.

Acceptance Criteria

1. User must be able search for courses by one or more keywords.
2. User must be able to see the courses and classes periodically being filtered while entering the search keywords.
3. User must be notified when there are no matching results for what he has searched for.
4. User must be able to cancel a search at any time.
5. User must be able to combine the search with the filters to narrow down the results even further.
6. User must not be able to search for any inappropriate words [listed elsewhere].

In the above example, this style of individual AC presented as line-item statements are often referred to rules-oriented acceptance criteria, meaning that we are declaring what the end result should allow the user to do. This is why I said earlier AC defines DONE: when these AC are met, the story has been designed and implemented successfully.

(Note that there is another popular style of writing AC: scenario-oriented acceptance criteria, or BDD acceptance criteria. However, I will cover that in a separate article.)

Now we’ll talk about some observations derived from this example:

Refer to the user

In the first line of the narrative, the WHO is declared as one or more users.  Recall that I advocated for at least referencing the user role, as developers can easily map the feature to the roles and permissions structure in the application.  So when we write acceptance criteria, we refer the “user” because the user role(s) were already declared.

In cases when the narrative refers to multiple roles, we still use “user” in the AC to convey it doesn’t matter which role the user is coming from.

Acceptance criteria define outcomes

Right up front, you’ll see these rules-oriented AC explain what the end results looks like. They don’t elaborate about HOW the feature will be designed or developed. They focus on what the user must be able to do, since every criteria item begins with that very phrase (though not a hard and fast rule). Rules-oriented AC are excellent because they can be written early, ahead of any UX design, and if written properly, allow the designer the freedom to create possible experiences that fulfill the criteria without being dependent upon conceived notions of what the final UX will look like.

Rules-oriented acceptance criteria does not depend on or describe UX

Building upon the previous point, acceptance criteria can easily begin to explode to multitudes of items when AC describe the user experience design. For instance:

1. User must be able to see the word “Search” next to the search bar.
2. User must be click on the search field to enter keywords.
3. User must be able to enter search keywords in the search field.
4. User must be able to click on the “X” to close the search field.
5. Use must see search results in the results area after entering keywords.
6. User must see search results change after removing a search keyword.

I stopped after 6, but this could have gone on forever. The difference between this set of criteria and the earlier set is that here I am effectively describing the UX, whether actually designed, or just designed in my head. All kinds of negative side-effects manifest with this AC:

• If the UX was already designed when this was written, I will have to periodically sync up this AC with any changes that occur to the UX between now and development. A new UX means complete rewrite of my AC.
• If the UX wasn’t already created when this was written, UX may feel constrained into one model of solving this problem. There is no design freedom because the this AC describes exactly one experience.
• The QA engineer now has been micromanaged into performing their work.

Acceptance criteria define the boundary of scope

AC define what is the measure of success for the user story. The outcomes described must happen, and therefore the AC ultimately sets the scope boundary. Any designed or developed functionality that doesn’t directly contribute towards hitting the acceptance criteria are extraneous and probably don’t belong in this user story.

Acceptance criteria can describe negative / exception criteria

Most of the acceptance criteria should be written in a positive form, meaning that we’re describing what the user must be able to do. Inversely the negative or exception form is what the user must NOT be able to do. It’s usually not as legible when a majority of AC are written this way because it implies that the reader can derive the feature’s capabilities on their own. In addition, writing in this way can often bleed into defining the user experience, which I’ve already covered as a trap.

Negative AC though does have its place.  I’ve found it most useful to define restrictions per certain policies of the application or company. Here are a couple random examples:

• User must not be able to change his password to a previous one.
• User must not be able to login from two devices at the same time.
• User cannot search for any foul language keywords [listed somewhere].

Acceptance criteria is consumed by your entire agile team

Though one person may be responsible for writing the initial user story and acceptance criteria, there are many consumers of that user story and AC, so spending enough time to solidify the AC is extremely important. Here’s some people who consume user story AC:

• Product Manager: Assuming the product owner is a separate person, the product manager reads the AC to ensure success is aligned to what the user feedback is telling her.
• Product / UX Design: UX designers refer to the AC to ensure their designs address the user’s goals, and doesn’t stretch beyond the bounds of DONE.
• Development: Developers use AC to gauge what is required to support success in the implementation, both in code logic and data structures. And all development and QA effort estimations are based on AC, so when missing, estimations will be off.
• Quality Assurance: These folks are responsible for ensuring the quality of the feature, and AC is the measuring stick to assess it.
• Stakeholders: Anyone else in the organization who has a stake in a feature should be able to read AC (and the user story as a whole) to understand what it will do.

Thus, if the AC are not adequately flushed out, ambiguity surfaces, many people are misaligned, and chances are the feature will be deployed with a high probability of rework…delaying final delivery.

Our methodology: when we write acceptance criteria

When we begin to describe a feature, immediately after the narrative, we begin to write acceptance criteria in the manner shown above, enumerated line by line.  Note that this happens before any UX designs have been created.  As our goal is to define what success and done looks like for this feature, we can review both with our client, UX, and development team before any effort has been put into design or development.  As we have already spent time understanding the product with our client ahead of time, our list of AC is generally pretty close to the final version, though sometimes modified as we review together as a design team.

Now, while the user story transitions to UX design, then development and software QA, everyone along the way from design to the stakeholders who sign-off will measure the feature’s doneness against the plainly written, easily readable acceptance criteria front and center in the user story.  The more we do this, the easier it becomes, and the faster we can deploy features correctly the first time.

So with the user story’s narrative and AC list written, we move into product design sprints and flushing out user story scenarios.   Stay tuned…