Validation

Validation is crucial to creating a proper test in Cycle. Validation can be understood as checking the accuracy of an action done by a user or system. In Cycle, we add Steps to a Scenario to add validation to a Scenario. Keywords can be used meaningfully to set up a structure for validation.

Review of keywords

The keywords Given, When, Then, And, and But can not only aid with the readability of the Steps in a Feature, but they can also provide a solid structure that supports validation.[^1]

Example of using keywords for a purpose

Note that these are not valid Steps, but are created to illustrate the concept

Given my system is in this expected state When I perform this specific action

Validation Steps start

Then I expect to see this outcome And I expect to see this outcome too But I don't expect to see this outcome

Validation Steps end

Go to the Keywords page for more information on using keywords for the purpose of validation.

Why is validation important?

Without adding validation Steps to a Feature--in other words, checking that when Cycle did what you asked, the result is what you expected--you can easily run into false positives or misleading results.

Imagine you are testing a website, and you need to know that a given URL is valid, and that it opens up the page in the format you are expecting. The following Feature would lead to "all green" (i.e., sucessful) results in the Output Pane in Cycle:

While the above Feature would execute successfully, what is it really testing? It is testing that Cycle opened the web browser, navigated to the CycleAutomation webpage, hovered over an element, and clicked it. It isn't testing anything about the results of those clicks, such as what text is on the page that loads. The above script would not catch an error with the URL being mapped to the wrong page, for example.

In order to test something appropriately, we need to consider what we are testing, and then validate our actual results against our expected results. In the Feature below, we've added Steps that validate the results of our clicks. We are testing that the appropriate page loads with the text we are expecting.

Feature: Test Features Page Scenario: Features page Given I open the "Chrome" web browser When I navigate to "http://www.cycleautomation.com" in web browser within 30 seconds And I hover the mouse over element "text:Overview" in web browser And I click the "FEATURES" link in web browser within 500 ms Then I see title "Features | Cycle" in web browser within 2 seconds And I see "SEE WHAT CYCLE CAN DO" in web browser within 2 seconds

The validation example above is very simple, but the concept can be generalized to all forms of testing. Imagine you are using Cycle to add a new user to a database. If you simply executed the Steps to add a new user, Cycle could easily show that 100% of the Steps passed; however, perhaps there was a database issue and the user wasn't actually created. Without validating that the user was actually created in the database, we can't be sure that our test truly passed--we only know that Cycle successfully executed the Steps.

Validation Steps

Validation can be done by checking that something exists in a database, verifying that text, images, or elements exist on a webpage, verifying cursor positions in a terminal, checking images in a desktop application, and more. One of the most common types of Steps used for validation is the "I see" group of Steps. Steps that begin with "I see" are inherently used to check that the user is able to "see" something on the screen, presumably after a certain action was taken in the system or process. Similarly, Steps that begin with "I verify" are used to check whether something the user cannot view on the screen is true.

[^1:] Cucumber Open Source Repository, Github, s.v. “Given When Then,” (accessed August 14, 2017), https://github.com/cucumber/cucumber/wiki/Given-When-Then.

Review of keywords​