Transcripted Summary
Here is an example SAP app that lists a bunch of electronic products.
Our job is to write tests that check the following:
Number one, check if all the products are listed correctly in this list on the left side bar..
Specifically, we need to check if the item’s label, image, price, currency, the availability status etc., are all shown properly.
Next see if the products in the right-hand-side pane are displayed properly.
But note that the carousel is dynamic, every time we come to this page, it shows a different image.
Also note that the featured items are also dynamic.
Every time we come to this page; different items are randomly displayed.
NOTE
As you can imagine, these things are impossible to test. This is actually also true in the modern way, but at least in Applitools, we can use something called an “Ignore Region” to ignore these randomly generated dynamic sections of the page.
Next, if we select a product, we need to see if all the details of that product are displayed.
Next, if we add a product to the cart, and open the cart, the item should be displayed in the cart.
And lastly, if we try to add an item that’s out of stock, then we should get an error message saying that it’s currently out of stock.
# Here is some sample legacy code that tests the above 5 tests.
This first test productListTest opens the app, and then simply checks all these details of a single item.
Details such as label, price, currency, availability and so on.
NOTE
Now, I have not written the code for all the items in this list, but I would imagine that this would blow up the code size by 10 times in order to store all the details of all these items. This would be the case even if you had some kind of a function that validates all these things for all the items.
Now if you look at the DOM, it’s very verbose and it’s very complex.
This is because of the SAP’s UI framework.
Now every time a new version of the framework shows up, the ids, class names, etc., could be completely different. They may make it less verbose. They may change how these ids are constructed; they may also not even use these ids anymore.
You as a QA engineer are basically out of luck.
You have absolutely zero control because this is not even your own company who is building this UI framework. It’s SAP!
So, as you can imagine, this can easily set you into a vicious cycle of code maintenance and false positives.
I wish you could avoid this completely, but as you’ll see soon, with the new approach, because you are taking screenshots, you can at least minimize the dependency to a great extent that’s easily manageable.
Next, the productGridTest checks just a single item in the grid.
Again, there is a lot of validation code to ensure all the details exists.
Next the productDetailsTest checks if all the details with the product are displayed in the details page,
So, this code selects an item to open the product.
And then we check if all the labels, and the corresponding values, the pictures, the availability information, and so on, are all displayed in the product details page.
Next, the addToCartAvailableItemTest runs the 3rd test.
It selects the 1st available item…
Then it clicks on “Add to cart” button.
Then it opens the cart.
And then we write a lot of validation code to again check if all the details of the item are displayed while it’s in the cart.
The next test addToCartOutOfStockItemTest tries to add an out-of-stock item to the card.
So, it selects this 2nd item that’s out-of-stock.
Then clicks on “Add to cart” button; then opens the cart.
And then we check if this “Confirmation dialog” is displayed. And we write all these validations to check if has the title, message etc., are all displayed.
Just to summarize, we just saw details of 5 tests.
We had to heavily depend on locators for every small thing. And we know that these locators are very verbose and can change at any moment, but still we had no choice.
The worst part is even if you write all this code for the first and second test, we could not write tests for all the items,, because that would have blown up the code base by 10 times. And that is the current state of things in many organizations.
# Now let’s switch the modern Visual AI way and see how simple things become.
In the first test, we simply take a full-page screenshot of this list.
@Test
public void productListTestNEW() throws InterruptedException {
// Start the test
eyes.open(driver, "SAP App", "Products List", new RectangleSize(1600, 800));
// Check the product list
eyes.check("Product List", Target.region(By.id("container-cart---category--page-cont")).fully());
// End the test
eyes.closeAsync();
}
One thing to note here is that, because it is a fairly long list, Applitools takes multiple screenshots as it scrolls down and then stitches them to make a single screenshot and sends it to our server.
So, you’d see it will take about 5-10 seconds to do all that.
It again depends on how long the screenshot itself is, but for most tests, you won't see this delay because you are simply taking screenshots of what is displayed within the viewport.
But in general, the best practice is to run tests in parallel to get the results as soon as possible
Optionally, if you were to use the Visual Grid, then also this would be significantly faster.
Instead of scrolling down things and then stitching things and then uploading it to our server, we simply take the dump snapshot and upload it to our visual grid server.
The visual grid server then asynchronously renders this page and then sends that to our AI server.
And because our visual grid server renders all these images asynchronously and also in parallel, this will only take a second or two.
Moving on…
The second, third and all the remaining tests do the same thing.
@Test
public void productGridTestNEW() throws InterruptedException {
// Start the test
eyes.open(driver, "SAP App", "Products Grid", new RectangleSize(1600, 800));
// Check the Product Grid
eyes.check("Product Grid", Target.region(By.id("container-cart---welcomeView--page-cont")).fully());
// End the test
eyes.closeAsync();
}
@Test
public void productDetailsTestNEW() throws InterruptedException {
// Start the test
eyes.open(driver, "SAP App", "Products Details", new RectangleSize(1600, 800));
// Open the 1st item (Available)
clickOn("__item0-container-cart---category--productList-0-content");
// Check the product's details page
eyes.checkWindow("Product details page");
// End the test
eyes.closeAsync();
}
@Test
public void addToCartAvailableItemTestNEW() throws InterruptedException {
// Start the test
eyes.open(driver, "SAP App", "Add Available Item to Cart", new RectangleSize(1600, 800));
// Open the 1st item (Available)
clickOn("__item0-container-cart---category--productList-0-content");
// Add to cart
clickOn("__button9");
// Open cart
clickOn("__button8");
// Check the Cart page
eyes.checkWindow("Cart page");
// End the test
eyes.closeAsync();
}
@Test
public void addToCartOutOfStockItemTestNEW() throws InterruptedException {
// Start the test
eyes.open(driver, "SAP App", "Add out-of-stock item to Cart", new RectangleSize(1600, 800));
// Select the Out-of-stock item
clickOn("__item0-container-cart---category--productList-1");
// Add to cart
clickOn("__button9");
// Check the warning message
eyes.checkWindow("Out of stock warning");
// End the test
eyes.closeAsync();
}
They use Selenium for navigation, and then take a screenshot.
And once the tests are run, you’ll end up with screenshots like this.
This is the screenshot of the 1st test.
This is the screenshot of the second test.
But remember that the carousel and the featured items list are dynamic.
So every time you take a screenshot, these items change.
The simplest way to overcome this is to simply add an Ignore Region.
This is how you’d do this.
This will tell our AI server to ignore this region of the page when it compares screenshots.
Now, this is the screenshot of the third test.
This is the screenshot of the 4th test.
And lastly, this is a screenshot of the 5th test.
Hopefully this gave you a really good idea about the advantages of writing functional tests using Visual AI.
