Running Tests from the Command Line

cycle-cli executes a Feature File, playlist, or group test and reports the result to the console. Use it wherever the Cycle desktop app is not available, such as a build agent or a scheduled task.

This page covers the invocation form and the parts of a run that need explaining. For the full inventory of commands and parameters, see Cycle CLI Commands and Parameters.

Running a test

1. At the command prompt, navigate to the directory where the cycle-cli.exe file is located (typically C:\Program Files (x86)\CycleLabs\Cycle).
2. Authenticate, if you have not already done so on this machine.
   • On a workstation, run cycle-cli auth login once. Cycle stores the session, and later runs reuse it.
   • In a CI/CD pipeline, pass client credentials on the run itself with --clientid and --client-credential. Inject the credentials as environment variables for security.
   • To sign in from a token instead of a browser, run cycle-cli auth login --token [arg]. This also stores the session.
   • See Running Cycle CLI in Pipelines for all three paths and for how to obtain the credentials.
3. Run cycle-cli run, then any optional parameters, then what you want to execute: a Feature File, a playlist, or a group test.
   • Parameters have to come before the target. Cycle stops reading parameters at the first file name, so a parameter typed after it is treated as another file to run rather than as a parameter.
   • Name more than one Feature File to run them in sequence.
   • By default, cycle-cli looks for a .cycuser file named after the user it authenticated as. For example, if you signed in as j.doe@cyclelabs.io, it looks for a j.doe.cycuser file. If you use a more generic .cycuser file in your pipeline to store settings required for your pipeline tests to run successfully (for example, jenkins.cycuser), you can use the -u (--user-profile) argument to specify the .cycuser file to use. A client-credentials run authenticates against the subscription rather than a person, so it has no username to derive a .cycuser file from. Pass -u in that case.

Example 1: cycle-cli run --output-directory C:\Cycle\Output --project-file MyProject.cycproj webtest.feature

Example 2: cycle-cli run --project-file MyProject.cycproj terminal.feature web.feature sql.feature

This would run each of the three Feature Files sequentially.

run also accepts a Blueprint, which is data setup rather than a test. cycle-cli run data.yml runs the file as a Blueprint, chosen by the .yml or .yaml extension, and it takes its own parameters rather than the ones on this page. See Running a Blueprint.

Prompting steps

If your Feature File includes steps that prompt the user, these steps are skipped.

Existing scripts that omit run still work

Older commands wrote the parameters and files directly, without naming a command: cycle-cli --project-file MyProject.cycproj webtest.feature. Cycle still accepts that form and treats it exactly as cycle-cli run, so pipelines and scheduled tasks written before commands existed keep working untouched. New commands are clearer with run stated, because it makes room for the other commands on the same binary.

Running cycle-cli in parallel

Applicable for Cycle 2.14 and beyond

Cycle uses an embedded H2 database to store execution results for reporting. This embedded database creates files in your APPDATA directory with a .db extension. These files are deleted automatically when Cycle is gracefully shut down. To handle a situation where any .db files were orphaned due to a forcible quit of Cycle, at start up Cycle tries to delete any unused .db files. This can lead to challenges when trying to run multiple instances of cycle-cli in parallel, where one instance of cycle-cli attempts to delete the .db files used by another instance of cycle-cli on the same machine. To avoid this situation, cycle-cli supports a --skip-initial-purge argument. When using this flag, cycle-cli does not attempt to delete any .db files at start-up, avoiding potential conflicts between multiple instances of cycle-cli.

Deprecated but still required

--skip-initial-purge is deprecated and prints a warning. It remains the supported way to run parallel instances until the underlying database change ships. See Deprecated parameters.

Project file parameter

This can be either the project directory or the .cycproj file.

For example, if a user has a project in c:\users\MyUser\myproj, either c:\users\MyUser\myproj or c:\users\MyUser\myproj\myproj.cycproj would be acceptable arguments to follow the --project-file parameter. This can be an absolute path or relative path. If this argument is not used, cycle-cli looks for a .cycproj file in the current directory. So in the command line, if the user changes directory to MyUser\myproj, they would not have to use the --project-file argument.

Cycle CLI results

cycle-cli writes results to the console as execution proceeds. Every line begins with a local timestamp, and nested steps are indented.

Each step reports its step text, its status, and how long it took:

10:14:22.913 I navigate to "https://example.com" - Pass - 842 ms
10:14:23.401 I click "Submit" - Fail - 488 ms

A failing step is followed by its error message, and by a stack trace when one is available.

While a step is still running, Cycle rewrites its line in place with the elapsed time so far. --no-tick turns that update off, which is worth doing when the console output is captured to a log.

Each Feature ends with a line carrying the Feature name, its status in brackets, and the elapsed time for the whole Feature:

10:14:31.204 Feature: Place an order [Pass] - 9153 ms

A status is Pass, Fail, or Skip. A step inside a conditional block reports True or False instead of Pass or Fail.

Any of the reporting formats available to the main Cycle client can be generated if the respective settings are applied.

Replicating settings

In many cases, cycle-cli is run on a different computer than Cycle itself. It is important for teams to carefully control what settings are in use when running cycle-cli. It is possible to replicate the needed settings by specifying a project file for your cycle-cli execution. All of the pertinent execution settings are included in the .cycproj file.

In addition to this, if there are a few specific settings that need to be overridden for a cycle-cli execution, they can be saved as "key":value pairs in valid json objects and loaded in for a given execution using the --settings-file parameter. Use --settings to set individual options directly on the command instead.

Defaults when no settings are supplied

Passing none of -u, --settings, or --settings-file leaves a run on Cycle's defaults:

• Results are only sent to the report service.
• No other reports are generated and no data is sent to the data store.
• No web driver or winapp driver locations are set.
• No stored credentials exist.
• No custom output directory is set.
• No KnownHostsLocation file is set.

A pipeline that expects a local report, a data store write, or a web driver has to supply the corresponding setting.