Transcripted Summary
Now that we've had some fun performance testing, mobile and web clients, let's go to the server side of the equation and I want to start out by looking at load testing tools. There are a lot of tools to choose from. The Wikipedia article on load testing lists 19 and I know for a fact that there's more that are not on this list.
**What are the things you want to think about when you're choosing a tool? **
Open source versus commercial
If your company already has a license for LoadRunner for example, that is probably the tool that you'll want to use. Whereas if you're just getting started, you might want to pick up an open source tool to begin with. And there's a lot of tools that you can start out open source and then move into an enterprise version of that same tool.
Local versus cloud
Are you expecting to run all of your test within your local network or are you outsourcing it to some kind of cloud provider that's going to do the running of the test and give you a report?
UI vs Script
Are you more comfortable with tool that has a UI or are you okay with writing a script for your tool in pure code? Something like JMeter which has a graphical interface might be better for somebody who's not as familiar with coding. But developers and other people who are comfortable with code might prefer Gatling or another tool where you can just write scripts directly.
Integration with other development tools
If you're in a Visual Studio environment, then it makes sense to use the Visual Studio load testing tool that's available. (Note: Visual Studio will be dropping support for load testing after 2019.)
Also, if there's already expertise in your company on a certain tool, then by all means leverage that expertise. So, I'm not going to recommend that you use one specific tool and that's the only tool to use. The important thing is to understand what's out there and understand what fits best into your situation.
I do however want to look at a couple of very popular open source tools just to give you an idea of what these load testing tools can do.
And I'm going to be using a local instance of the Restful Booker API to do the test. If you took my course on exploring APIs then you're very familiar with this particular API, which is basically a reservation sort of service. The test scenario is that we have frequent booking lookups and less frequent addition of new reservations. So, it's a very simple scenario just to show the capabilities of these tools.
Apache JMeter
The first tool I want to look at is JMeter. This is what I'm most familiar with. This is what I personally use for most of the load testing that I've done. It's a GUI-based tool and it's written in Java and it was first released in 1998 so it's been around for a long time, but it's still an active and the latest release I believe came out in February of 2019.
So, because it's been around a while, it looks a little bit dated, but it's actually very powerful. One of the reasons it's so powerful is because it has such a rich plugin ecosystem. You can use JMeter for HTTP calls, FTP calls, web sockets, database, Redis. There's even a web driver plugin for Selenium. Basically, any way that a client and a server can communicate with each other, it's probably covered by a JMeter plugin. And if it's not, you can write your own plugin pretty easily if you're familiar with Java.
JMeter also supports the Groovy scripting engine, which is a scripty version of Java essentially. And with that you can add a lot of complexity to your test that might not be available just in the drag and drop UI interface.
JMeter also supports record and playback so you can gather information about your websites that way.
Jmeter also supports distributed testing. So, you can have a central machine that is controlling a lot of worker machines to generate load within your local network. And that can be very handy for running load tests locally.
Let's take a look at JMeter itself.
Here we are in JMeter and this script will be available as a resource. So, if you want to install JMeter and take a closer look at it, you certainly can.
And this is the somewhat dated looking interface and the plugins are here in the plugin manager. You can see this is just what I have installed — MongoDB, LDAP, Junit — and there's a lot more that's available that haven't been installed yet. JVX, there's Kafka support, there's Perfmon, there's WebSockets —there's all sorts of things. Selenium WebDriver support is something that I keep meaning to look at, but I'm sure it's very useful.
So, very rich ecosystem there.
The way that works is you have a test plan and then you can create sort of local variables for your test plan and then you create these thread groups
Each thread essentially represents a user and this particular thread group has a hundred users and it runs for 90 seconds. And within this thread group I have these HTTP requests, which is going to the host and looking at the booking, getting it by name, id. So, any HTTP requests you can put together in one of these samplers or you could add more.
This is how you add things here and you can see that there's a lot here.
And there's a lot of timers so you can do a lot of very detailed shaping of your traffic to make sure that it matches what you want to see in production. And then I have another thread group that is creating the booking and I have another timer here to make sure that each thread is only creating one booking a second. So, the throttling aspect is also really nice.
And then for looking at the test results, we have also many, many possibilities. All of these listeners are here and more that I haven't installed.
Let's take a quick look at what it looks like when it runs. When I run it in the UI, I can see as I'm running what's going on with it in all of my listeners. Here’s the throughput and et cetera. I think I didn't have the service started and that's why everything is failing.
Okay. So, I didn't have the service started and that's why the previous test was 100% failure. But that's okay because what we really want to do is not run our tests in the GUI.
We need to run them on the command line for maximum performance. I can do that pretty easily.
Now the test is running and when it's complete we can look at the results. So, now the test is complete, and you can see what's happened here. No errors this time. And we can look at the results in JMeter itself.
So, any of these listeners, you can load a CSV file in to see the results.
We can see that as more things were added in, the performance got worse, which is what I expected because it's not a very good service. It's meant to be bad.
So that's JMeter and this is obviously not meant to be a detailed tutorial on how to use JMeter. I can actually do an entire class just on JMeter if there's interest in that. But this is more of an overview.
So now let's move on to the next tool.
Gatling
The next tool I want to look at is Gatling. I'm not personally as familiar with this tool as I am with Jmeter, but I know that it's very popular with a lot of different companies.
Gatling is a Scala-based tool, and Scala is a scripting language that runs anywhere. It runs in the JVM so it's not tied down to Windows or Mac or whatever. It can run anywhere. It's first release was in 2012 although I would say it's really in the last two or three years that it's gotten really popular.
And it has a pure scripting interface. So, there's no UI where you can pick things from a menu and drag them into position. You just open up the IDE of your choice and write a Scala script and that's how you do Gatling.
Gatling also supports record and playback, which might be useful just to get an initial start on your script.
And because it has a more modern architecture than JMeter, it can actually support a higher load. So, every instance of Gatling can produce more load than a single instance of JMeter.
Gatling also has an enterprise version if you want the enterprise support. And for more information, just go to gatling.io.
I'm using Visual Studio code as my IDE with the Scala plugin. And we have these objects that are essentially equivalent to the thread groups that we saw in JMeter. So, this is my browsing object where I get the bookings and then I get them by name, et cetera.
And this is the booking object that's creating the reservation. I personally find this format to be easier to read right off the bat than looking at a JMeter script. And here's where I'm setting up my variables.
And then the objects that are going to actually do the job itself. So, I have a hundred users that are just looking and 30 users that are making bookings.
Once you have a script, you need to run it in the command line. You can't obviously run it here.
I have Gatling running in a terminal window and you can see my restful booker simulation is there and I can just select it to run (Select run).
Now it's starting up. And now it's running, and it'll update every 10, 15 seconds just so that I know that it's still going. So, our script is almost done.
And we get a basic idea of what happened here. But then what's really nice about Gatling is that it gives us this HTML report. So, let's take a look at that. We just copy that and stick it into a browser.
And we can see that it has a lot of information for us on how our test did. And it looks very pretty. So, this is sort of a nice instant feedback that you get from Gatling. We don't have to put things into different listeners and so forth.
So that is the basic runnings of Gatling.
Generating Load
Once you have your script figured out, of course the idea is to run a lot of instances of your scripts so you can generate hundreds, thousands, maybe even millions of users for your application.
And the ways to do that you can either do it locally within your network. If you have an internal cloud available, or if you have a lot of virtual machines, this could be useful for a smaller service. It might be even able to generate enough load for an initial test on your own laptop.
The next step is going to the cloud. So, you could use a cloud provider, a company that works in the cloud such as Blazemeter or whatever. Or You could just put your own JMeter or Gatling instances out in EC2 or Azure or whatever you're using and run your test that way. So, the cloud makes it a lot easier to generate a high volume of requests.
And finally, third party vendors really come in handy for very high load. If you need to generate millions of requests a minute, you probably want to look at a commercial vendor such as Flood or BlazeMeter or use something like LoadRunner to do that.
But we'll go into more detail about designing your load tests in the next chapter about load testing strategy.
