Debug your tests

During the development of your automation, you are mostly busy with designing your test case, finding objects on screen, verifying test case flow, checking test data, etc. To accommodate this process in JOSF, you are able to start a debug session. Also, when your test case fails, debugging the individual test case helps finding issues faster!

Debug session

Starting a debug session, starts a test context in which you can execute test steps on by one. It creates a test context, in which it can open a browser and store variables.

The key difference between a debug session and regular test executing, is that a debug session won’t stop during failures, for example when objects aren’t found or when verifications fail. This makes it your go-to tool when maintaining or developing new test cases.

Breakpoints

For each step in your test case, you can add a breakpoint. Breakpoints use useful when test cases fail and you need to investigate what is going on. Adding a breakpoint to a test step will execute a test case up until that certain breakpoint, from whereout you can continue your investigation.

Note that breakpoints are ignored during playbook runs or when you enable the “skip breakpoints” toggle.

Test context

When a test case is started, before anything, a new test context is created. The test context stores important information about the started test case such as the test status, variables, API responses and web browser status

Test status

During a test run, each test step is executed. When a test step fails or runs into any error, the step will hold that status. If your test run is not a debug session, the failed or erroneous status will be set to the test context. Any following executed test step will then be skipped, as a previous test step failed to correctly execute.

Note that during debug sessions, the failed or erroneous status of a step will not be given to the test context, to continue the session.

A test step can fail or run into an error for different reasons. The most common reasons are;

A verification failure: e.g. your expectation is not equal to actual value on screen.
- JOSF will give a message stating how the expectation differs from the actual found value.
An object that should be on the screen or a node should be in an API response, but is not found with the given strategy:
- JOSF will mention that the object is not found, given the specified strategy.

Variables

One of the most important concepts within JOSF and test automation, are variables.

Variable values can be set manually with static data or set by extracting values from an object on screen or from a node in an API response. The lifespan of the value of the variable, the length of time it holds its value, is ended by two events;

The end of the test execution, and by extend the end of the test context; and
When the same variable name is overwritten by a new value.

Using variables requires the variable syntax, which starts with a ${ and ends with }. Assuming we have a variable named order id, using the value in that variable requires us to write ${order id}. This tells JOSF to lookup order id in the variable list from the test context and use the set value.

An example; Setting and using variables

In this example, assume our application under test is a web shop, in which we’ve ordered a product and the order page is visible, containing our order ID. This ID is something we’ll use to look it up in the application’s back-end to process the order. To accomplish this, we’ll use the set keyword<<SET ACTION REFERENCE>>. To find the appropriate objects in the web browser, we are using the XPath strategy. More about object strategies can be found here.<<XPATH REFERENCE>>

#	Action name	Object	Data
…logic of test steps to get to this point…
1	set	//span[@id=’order-id’]	order ID
…logic to login into the back-end…
2	type	//input[@id=’search-order’]	`${order ID}`
3	submit	//input[@id=’search-order’]

First, we set the value of the object into a variable named ‘order id’
Then, we use the value of the variable, by typing it into the search field, with the variable syntax.
Finally, we use the submit action on the search field, so that the search is executed.

Variables can be used in most places (object, data, API fields) and can be combined with text inline.

#	Action name	Object	Data
2	type	//input[@id=’search-order’]	Order id: `${order ID}`
3	click	innerText=`${order ID}`

Note that variable names may all characters (including spaces) except for $, { or }.

Note that variables are case sensitive. A variable set as order id, won’t be the same when using it like ${order ID}. To avoid confusion, we recommend avoiding capitals all together.

Note that when a variable is used before it is set, JOSF will not throw an error. Instead it uses the literal variable text, so ${order ID} will remain ${order ID} instead of the value of order ID.

Note that during debug sessions <<DEBUG SESSION REFERENCE>>, the test context will remain open until manually stopped, and therefore will retain all variable values set in that context.

API Responses

When executing an API call from a test case, the response of that call is stored in the API responses list, within the test context. Any API Action that is executed, will be performed against the latest API response.

Web browsers

If JOSF executes a web action for which a web browser is required, JOSF starts a browser by the given configuration. Once the test context is closed, any open browsers will be closed with it. Each browser starts fresh, without any cookies or sessions. This means that session/test case related information (like login data) is not transferred between test cases.