Transcripted Summary
Now we've seen a few examples and dug right into the code, and we've looked at some heuristics for doing code reviews and testing by looking. But we really need to take a step back and make sure that we all understand some of the fundamental concepts in computer programming. And I like to call this one, "Coding, how to fry an egg."
And you might be wondering, “Well, what does frying eggs have to do with coding?”
This really comes from back in community college on the first day of one of our programming classes, I had the professor come in and ask the class, “Class I need you guys to write an algorithm to fry two eggs.” And we were all very puzzled, fry two eggs? How can we do this with a computer?
And for us we were already jumping to what was a coded solution. Whereas he was asking us for an algorithm, which is just a step-by-step set of instructions to solve a given problem. With that, the idea of a recipe is a very good analogy that helps us to understand and illustrate the different fundamental structures within computer programs.
Let's get started.
So, how to fry an egg?
- Well, the first thing we need to do is to retrieve the different ingredients and cooking utensils at the beginning of the recipe.
- And then we can melt a teaspoon of butter into a frying pan.
- And crack an egg and pour that into the frying pan over heat.
- And once the egg is cooking, we have to make sure that we use the spatula to prevent sticking.
- And then comes the million-dollar question, is the egg to preference. We think about the consumer, right, the person who is going to be eating this egg and how they like their eggs. And we have to make sure that the egg is cooked to their satisfaction.
- And if it's not, we continue cooking the egg, right, and we go back and continue moving the spatula, preventing sticking, until the egg is finally finished.
- And then we can serve it on a plate with toast and we're done.
So, what does any of this have to do with computer programming?
Well as we said, computer take specific instructions and a recipe does just that — and there's actually three fundamental concepts at play here, that deal with computer programming.
The first is the idea of a sequence of instructions. We have a first step that flows into another step and flows into another step.
We also have the idea of selection, which is where we have a decision point, right, is the egg cooked to preference? Yes or no. And based on our answer to that decision, we either go one path or we go another path.
And last but not least, there's this idea of repetition. The idea that maybe as part of a decision, we go back to a previous step and we continue looping in that manner. Another word for that is iteration. We have these three fundamental programming concepts — sequence, selection, and iteration.
And now, we're going to start to look at, what are the different things that can kind of go wrong when we start talking about these different code structures, and how does testing fit into all of this?
Code Structure: Sequence
Sequences are your most basic code structure, where you just have one statement or instruction, followed by another, followed by another, all in this predefined order.
If we look at our example of a function defined as seq which takes parameters, x and y, we can see that here we just have one statement followed by another, followed by another.
And in trying to understand what is happening, we can examine each statement. Here z is being assigned to the value of x + y, and that calculated answer is being returned.
So, what could go wrong, it's just a simple structure?
Well, the first thing to remember is that several things could be wrong with the program, because whether or not you have a user interface in front of you or you have code, you also have to remember once you're testing that there's some business need that needs to be satisfied as part of this implementation.
What are your requirements?
The developer could have gotten many things wrong, if he didn't understand the requirements or if he just made a simple mistake. Here, maybe it wasn't supposed to be addition, maybe it was supposed to be multiplication.
Any statement in your sequence can be wrong, or perhaps there's other requirements that haven't been satisfied as part of this function, so there's missing statements.
Or, another thing that could be wrong and could really affect the output is if you have steps in an incorrect order. Maybe you have one step that is actually after, but it should be before another step.
What else about testing — are these some good tests for this program?
Here I'm showing you values of x and y that could be used as imputes to this function:
- (0,0)
- (2,2)
- (7,0)
- (0,9)
- (6,5)
Are these tests any good? Well, the first thing we don't really see any expectations, and as we just mentioned there needs to be some expectations based on the requirements of the application. Perhaps we have these expectations.
The first thing that we always have to do when we examine tests at this level, is to ask the question, where did we get these expectations from? Because if these expectations came from looking at the code, then the question becomes would these tests ever fail? In other words, we're looking at the implementation, we're taking inputs and we're basing our expectation off what the program is calculating. This is a very common mistake and common pitfall when people are doing unit testing.
That's why sometimes you’ve shipped a product to customers and customers are unhappy, but all the unit and integration tests are passing. And if we write tests like this, then obviously these expectations would only fail if someone changed the code. And that's not what we should be doing.
We should be looking back at the business value because only then we can see if these statements are even correct, because they don't match our expectations based on the requirement.
Let's assume that this was indeed the application expectations to add these two numbers. The question remains still, are these tests any good? We have a test (0,0). That seems okay. There's a boundary and zero is that special case that we always want to check for, now these here, we're checking it on both the inputs and the expectation.
Now we see another test (2,2). And these seems sort of very similar to this test.
And it begs the question, do we really need both of them or could we optimize this?
But there's something that's also very important in this particular context, about both of these tests that make them not very good, especially if we put them on their own.
Well, here if our developer did make a mistake and let's say for example, it was indeed supposed to be multiplication rather than addition, then these two tests even with that change, would pass. Because these two tests in this particular example have no discriminatory power. In other words, 0 + 0 = 0, but also 0 x 0 = 0, and the same goes for (2,2).
And we want tests that are built in a way that if the application changes, especially when it comes to the functionality that the test can detect that type of chain. That's not so good in terms of not having discriminatory power.
Let's continue.
- (7,0) well this test is okay, because it sort of looks at the contribution of one variable in isolation, rather than the other.
- Same thing with (0,9) in how it does the same thing for
y, and we can see if any part isn't contributing. - And then (6,5) takes those two together. The tests aren't so bad, but we have to remember that we need to make sure that the test set that we have has discriminatory power.
Code Structure: Selection
Selection — recall that this is where we have a choice. Where there's some decision statement that needs to be evaluated, that can either be true or false, or yes or no, and depending on the value of that, determines which path we follow through the application. And so, there's several paths through the code depending on how those conditions are evaluated.
Let's look at some code.
- Here we have a function defined called condition and it takes a single parameter
x. - And now we have our conditional statements. It says that
if x > 10, then we'll doz = x + 10. - Else, we'll do
z = x – 10 - And then at the end, we'll return the value of
z.
What could go wrong here?
Well, the first thing that should come to mind is that even though there is selection now introduced here, this is still a sequence of statements. We still have one statement followed by another, followed by another, followed by another — no matter which path we go down. And so, anything that applies to sequence can also apply here.
In other words, there could be missing statements, there could be statements that are out of order, and so on and so forth.
Selection also introduces the potential for other issues with the code. For example, the developer could have gotten this wrong, with respect to the requirements. If you recall from our testing techniques, we can do boundary testing around this, to help us determine if we have made an “off-by-one error.” And so, what is that? That means that if the developer really meant instead of greater than, greater than or equal to, then testing around the boundaries, testing 9 and 10 and 11, would help to reveal such an off-by-one error.
That's one common mistake that can happen, when we start talking about conditions is that the relation operator might be wrong.
Of course, the values are variables on either side of this could have also been wrong. And so we have to start thinking about, especially, and being especially careful that when we start talking about selection, that we're paying close attention to these operators.
What are some good tests for this?
Well, typically when I ask this question, some of the first tests that people come up with are one test that satisfied this condition right, so maybe we can pick something like 20. And then another test that satisfies this condition, and for that we could chose something like negative 10.
Now, I'll ask the question, just because we've now covered these two conditions, are we done testing?
Because sometimes we look to cover 100% of statements, right. It's a common thing that we do in industry, we say we want to get 100% code coverage. Well, you can actually gain 100% code coverage with just those two tests that we mentioned.
However, there's still a lot of tests that haven't been done if we really want to have what's called good test coverage.
And for example, we mentioned earlier testing around these boundaries. Those are powerful tests because they help prevent and reveal certain types of mistakes, but they're not covered here even though we have 100% of statements covered.
And so I just use this to introduce the fact that when we start talking about code coverage, sometimes it can be tempting to try to maximize this number. However, we need to remember that real testing is about the variety of values that you choose to impute into the program that will make this program robust, and not just covering statement.
And things like robust boundary could be applied here to make sure that we really have a statement that is really solid.
All right. Any other tests?
I'll let you think about those and jot those down, but there's several other tests that come to mind here, because we've only covered some simple boundaries and some happy paths along these conditions.
Code Structure: Iteration
Iteration. This is by far our most complex control structure, because here there's sequence, at some point there's also a decision that is made, and so there's selection, and then we also choose to go back to previous statements, and that's your repetition or iteration.
Thinking about what could go wrong, we just need to remember here that again, everything that can go wrong with a sequence, or everything that can go wrong with a selection or conditions, can go wrong with iteration. But now there's some other aspects that we need to think about. Because since we're iterating and doing things, we have, for example, a loop that iterates over a list of items. And here we are saying for every L in our list, which meaning every item in that list, we're going to do some function and pass L to it. And then we're going to return some results.
There are several things that can happen here.
For example, maybe we didn't initialize our loop properly, or maybe we didn't finalize it properly when we were done. And you have to start thinking that anytime we're iterating, those are boundaries that could be off, and they could be off by one, they could be off by many.
And let's start to think about — What are some good tests? — when we start thinking about iterating over a list like this.
Well, we first want to test if we have no iteration whatsoever. Here we'd want to test an empty list to see what happens when we go through and we skip this all together, what is the result. You also want to look at one iteration, maybe more than one, something like two. And then from a business perspective, when we start thinking about business value, we want to iterate whatever the typical number of times this function is likely to be called. And this goes back to us thinking about how do we deal with positive testing? We want to make sure that positive tests get executed and executed early, because those are the most likely things that clients will do. There's a great risk if you release software and you haven't actually tested, the happy path or the most common or typical path that customers are going to do upon first getting this software. And then we can start doing other things like looking at the maximum number of iterations, and also going beyond that maximum. And so, a lot of things that apply to boundary testing can be applied to iteration.
Testing By Looking Lessons
Now, let's just summarize this lesson.- Code reviews can be an effective way to find defects.
- Looking at the code allows you to perform more targeted testing.
- Expectations should come from the requirements not code.
- Tests should have discriminatory power.
- Try just below, just above, and exactly on boundaries.
- Try less than, equal, and greater than for relations with variables.
- Try 0, 1, 2, typical, maximum, and maximum + 1 times for loops.
The first thing to remember is that code reviews are very effective. You can do a lot by just looking and using your intuition and your gut feel, and the sense of whether or not this program is confusing, if you think about the requirements whether or not, it's correct, complete, or if there's just inconsistencies throughout the program.
You can perform more targeted testing. By looking and taking that information allows you to come up with test cases that allow you to target specific paths through the code.
The expectation should really come from the requirements and not the code. And your test should have discriminatory power.
You also want to try just below, just above, and exactly on the boundaries. Now, with things like relations you want to try things like less than, equal to, or greater than.
Last but not least, you want to try zero, one, two, the typical number, as well as max and max plus one, in terms of iterations over that loop.
