Transcripted Summary
Now that we've learned about what visual testing is and why it should be an integral part of any QA framework, let's talk about Applitools. In this chapter, we'll show you how Applitools provides both functional and visual testing coverage. We'll also discuss other visual testing methods and understand how Applitools stands out in comparison.
Before we get into the Applitools solution, let's go over some other common methods tools use to try to catch visual bugs. The first method is DOM comparison. This method compares two DOMs and reports any differences between them. There are some key limitations with this approach. DOM comparison is similar to assertion based testing in that it does not take into account the rendering of the elements in the browser. Two identical DOMs could render differently causing DOM comparisons to miss visual bugs. Additionally, two different DOMs could render the same way causing DOM comparisons to flag issues that don't result in visual bugs. Due to these limitations, DOM Comparison is not suited to provide full visual coverage of a webpage.
Another method is pixel-to-pixel comparisons. Pixel-to-pixel tests are flaky and the tests will only pass if the images are 100% identical. Even with a shift in rendering by 1 pixel, pixel-to-pixel validation will disrupt your release cycle. Also, reviewing these test results can be cumbersome. It's unclear what the actual cause of a failure is. For example, Let's assume you added a header bar to the top of your application causing the rest of the elements on the page to be shifted down. In situations like this, Pixel-to-Pixel comparison will flag the entire page as being different when in reality, the only thing that's changed is the addition of a header bar. Because of this, P2P validation should be avoided since it provides flaky and unclear test results.
Applitools captures visual bugs with a different approach. Rather than basic pixel-to-pixel comparisons or DOM comparisons, Applitools uses a highly trained AI that mimics the human eyes. The Applitools Eyes AI is smart enough to understand if a difference is significant enough to report or if it can be safely ignored. It will sift through all the “noise” and give you clear and concise results.
To highlight this, let's see the Applitools Eyes AI in action. I'm sure we're all familiar with the classic "spot the differences" game. With a quick look at these images, we see that the candle has burnt out, a leaf was removed, a rock was added, a cloud is missing and much more. Additionally, if we toggle between these two images, we also notice that our image has rendered slightly differently and appears to stretch by a couple of pixels. Let's look at what happens when we do pixel-to-pixel comparison between these images.
As we can see, this method is flagging every single pixel on the page as a difference. It is essentially saying "The entire page is broken, but I can't tell you exactly why". What do we say to our web developers when they ask what caused our tests to fail? Pixel-to-Pixel comparison is not helping us identify what is actually different between these two images. Instead, it is forcing us to find those differences ourselves and wasting our time.
Let's compare this to the Eyes AI. Using the Eyes AI, we're Immediately able to see all of the important differences that we touched on earlier. We can take these test results and have a productive conversation with our web developers about these visual differences. We're able to say "Hey did you remove this cloud on purpose? Was this rock supposed to be added in? Did you mean to remove this leaf? and so on. This is the power of the Eyes AI, its ability to block out the noise, such as the slight rendering differences that caused pixel-to-pixel mode to highlight the entire image, as well as only highlight the differences that are interesting and useful. In addition to this AI algorithm that is trained to mimic the human eyes, Applitools has a number of other algorithms for visual comparison that fit a wide array of use cases. We will be going over all of these in depth in a later course but if you'd like to read more about them now, I've put a link in the write up below.
This chapter has been your introduction to Applitools and how it captures visual bugs. We learned about other Visual Comparison methods such as Pixel-to-Pixel and DOM Comparison and saw where they fell short. We then ran through an example that shows the benefits of the Applitools Eyes AI. We saw how it provides accurate and clear test results by using AI that's been trained to mimic the human eyes.
