Transcripted Summary
We're going to go through the who, what, where, when, and how of testing from the inside. What are we talking about? Well, testing from the outside, sometimes referred to as black box testing, is where we treat the program as a function from inputs to outputs. So, we really don't care about what's inside the box. We don't use any knowledge of the internal workings of the box. We simply take some input x and check the output f(x), and that's why it's called functional testing, because you're treating that program as a function.
What about testing from the inside?
In contrast, we're talking about what is commonly referred to as white box testing. Here, you're opening that box and looking at the internal structure of the software. We're trying to exercise the different programing and code structures that are present in the application. But we still care about inputs and outputs, so I don't want you to think that the correctness of the program, the correctness of that function, doesn't matter. But here what also matters is how well are we exercising that internal structure. So sometimes you can see that referred to as structural clear box or glass box testing.
Why should we care about this?
First thing, functional testing is just not enough. The medical field needs both physicians and surgeons. You can look perfectly healthy from the outside, and you can have high blood pressure. In a similar way, we can't just take an external perspective to determine the correctness of the software. We need to look at the internals, because ultimately that code matters. There's a lot of different aspects of the programming implementation that become the source of failures.
So, the technical aspects of the program are highly critical in determining correctness. A lot of the functional and performance issues are actually due to bad coding. And bugs can occur at any level of that implementation, all the way from the user interface through the service level down to that data storage layer.
Let's get into some of the advantages of testing from the inside.
Well, the first thing, it's really easy to find different types of input data that can help testing the application effectively. Because now you can see the different functions, you can see the different variables and inputs that they take, and you can now select proper data for exercising specific paths. By looking at that internal logic, you can be a lot more thorough.
And you can still use different testing techniques to select that data, but now you can actually see, well, hey, is there part of the internal logic that I haven't explored yet that pose a risk to the project?
Also, it facilitates writing smaller component level tests that provide early feedback on quality.
And so, it also draws some attention to bugs that can come up through specific coding patterns and technologies. As we mentioned before, technology can be the source of failure, and there's many cases where there are lower level or framework level bugs that creep into your application, and if you're not really exploring at that level, you'll never know or never find those types of issues.
What about some of the disadvantages?
As you can see, it requires individuals who are highly skilled in both design, coding, and testing. These are like your switch hitters, right? So, you can put them up to bat, they're very versatile, and they can bat in any direction regardless of which hand the pitcher is throwing the ball with. That becomes a harder skill to find, but at the end of the day, it also makes us — if we start to learn these skills — more marketable, especially in this day and age.
Now, large systems obviously it's very difficult or nearly impossible to search every section of code. Sometimes we're dealing with millions of lines of code and searching through that is not always very scalable if you're looking for errors.
In some cases, you have a situation where after you see a solution, you might get biased. And if you wrote this code especially, then you have a certain bias.
White box testing or testing from the inside is also very time consuming. And like most other techniques, it's not a silver bullet — it has blindness. That means that there's some issues that it will find and others that it can't find at all.
There're also some current trends that make us care about testing from the inside.
Testing is Dead
The first trend that we heard of was that testing is dead. And then they were like, "No, no. Testing isn't dead. It's the tester that's dead. Long live testing."
To get to the bottom of this, the idea is that some companies are really adopting this “No Testers” approach. They're actually saying something that we are glad to hear, which is that they want more and more for every role in your organization to care about testing and to care about quality.
And so, some of those traditional roles are being blurred. Everyone is an engineer. And this is better because other engineering disciplines like mechanical engineering and civil engineering don't have a testing role, so why do we have software testers? We should all be generalists and have some area of specialization.
And developers should do testing. And there should be group ownership throughout production.
Shift Left Testing
Another trend that's related is this idea of shifting left — doing more testing earlier and making testing an integral part of development.
In fact, this is what agile testing is really all about. Let's do validation of verification at every single stage and not wait in more of a waterfall fashion until the end to now think about testing.
And so, this idea of shift left and this idea about testing from the inside go hand in hand. In other words, we should be able to shift left from this waterfall testing approach into doing more testing earlier in development and design it alongside developers, writing unit tests, validating unit test, reviewing unit tests, and running them and executing them.
Continuous Integration, Delivery and Deployment
You've also heard about continuous integration, continuous delivery, and continuous deployment. We're living in this world where we want to ship things to production quicker. And testing needs to help enable that. So all this continuous integration and continuous delivery, and continuous deployment is only possible if we're doing continuous testing.
Testing is Evolving
And last but not least in terms of the reason that we should care is that testing is evolving. And as it evolves, how can we stay relevant? How can both developers and testers evolve along with this change in the testing world?
Well, the first thing I think for testers and quality engineers is that they need to enhance their technical skills. Developers need to enhance some of their testing skills. And it's great because we can find out what types of information as we work together are useful for the other role and vice versa. And this will just promote deeper collaboration and support.
All right, who should care about testing from the inside?
Well, I think we answered some of this already.
Quality/Test Engineers
First, the quality and test engineers should care. It's better to test something from the outside in, and the inside out. And so, a course like this helps us to do more thorough testing, and we can actually bring a lot of the information that we have from knowledge of the outside to the inside world with us, and we can also go the reverse direction.
Developers
Every developer by default should be testing from the inside. Sometimes I write a lot of articles and publish papers and I'm usually the first person to proofread my work. Now when I proofread my work, I give it to someone else to review, but I need to read it first, right? I need to take some sort of ownership of the quality of what I'm writing. Now of course, when I hand it to someone else, they find issues and that's expected. It's not because I suck at reviewing, but because I'm the author of that article, I have the story in my head and whenever I proofread it, I go over that story in my head and I don't really see some of those mistakes. But someone else who hasn't looked at that story before and looks at it with a pair of fresh eyes, now sees those mistakes.
But the idea here is that developers, even though they're developing the code, they need to be testing it as well. They need to take pride and ownership of quality from that point, and then we also need a fresh set of eyes, either other developers who are peer reviewing their code and testing with them or someone who's more of a testing role or a quality engineering role to be that second pair of eyes.
Now where does this occur?
Well, we're all familiar with the testing pyramid, but it doesn't mean that testing from the inside can't happen at the system level. In fact, any time we take a look at the DOM, or any aspect of the implementation of the user interface, we're no longer treating that system as a black box. We're now treating it as a bit of a gray box.
And so even though there's a big focus sometimes for black box testing or testing from the outside to occur at the system level, and white box testing to occur at the lower levels, there's nothing to stop us from using this information at any point throughout the pyramid. We can apply these techniques at all levels. However, you'll see more and more of this type of internal information being leveraged at those lower levels, integration and unit.
