How to Debug Automation Results

Reviewing a Failed Step

From the run page, click on a failed test's row to view details on why the test failed. Expanding the row shows a high-level overview of the failure with the following information:

Target screenshot: the screenshot you captured during test writing
Closest match: the closest image match the automation service was able to find
Freeze frame: a snapshot of the virtual machine during that step

To take a closer look and update the step, click Inspect and fix.... On the next screen, either select Update screenshot... to replace the step with the closest match, or Preview test to this step to make further updates to the test.

Click Open to view the test result in more detail or make additional changes outside of the failed step.

Test Retries

Sometimes, tests fail due to reasons including network, load, and intermittent bug issues. The Test Retries feature allows you to automatically rerun failed automation tests up to 3 additional times to attempt to pass the test. For more information on this feature, see Test Retries

Image Comparison and Update

For failed steps, you can investigate further by comparing the target and closest-matched images.

Click Inspect and Fix... on the expanded test failure.
On the next screen, scroll down to use the Swipe or Layered options to determine whether the closest match can be used.
When you determine what changes need to be made, either select Update screenshot... or Preview test to this step.

The Site Can’t Be Reached When Navigating to the Test URL

If you find the Rainforest virtual machine can't reach your website, make sure to whitelist all Rainforest IP addresses. See Which IP Addresses Do Rainforest Tests Come From? for more information.

Troubleshooting with HTTP, Browser, and Video Logs

With HTTP logs, you can view all HTTP traffic that passed to and from each Rainforest virtual machine (VM) during a test. Use the logs to examine request interruptions at the network level and see the order in which HTTP requests were made.

To view a test’s HTTP logs, browser logs, or a recording of the video results:

Click the Open button on the test's row on the run page
Click the 3-dot icon at the top right of the page.
Select “HTTP Logs,” “Browser Logs,” or “Video” to download the relevant file.

View the HAR and browser files in your text editor of choice, using this recommended tool or Google Chrome extension.

If you have any questions, reach out to us at [email protected].