Transcripted Summary
We need everyone on the team to collaborate for successful tests, automation and DevOps.
How can we make that happen?
For me, DevOps is a short way of saying: “Everyone on the team test continuously so that we can continually deliver small changes to production, minimizing risk, and delivering value at a sustainable pace”. That's kind of a mouthful, so DevOps is easier.
The right test automation is a key part of doing continuous delivery or deployment successfully.
Both test automation and continuous testing are key success factors as we've learned from the State of DevOps survey.
Testers, developers, operations specialists, business stakeholders, database experts and others on the team, are all needed to make this work. We need those fast feedback loops from our automated tests, and we need that fast feedback from production.
Teams can get started with automation and be fairly successful, but then they can run into problems. It's kind of hard to make that effort be sustainable and keep advancing towards being a high performing team.
Let's look at some pitfalls identified by the State of DevOps survey.
The survey found that Medium Performers fell into doing more manual processes and started accumulating technical debt, so they found it was a steep curve to get to the high performing level.
Be prepared to automate what can be automated, while continuing important manual testing activities like exploratory testing and budget time to manage your technical debt. This is a definitely a whole team effort.
The survey found that Medium Performers do well in terms of stability, but it just takes them much longer to get new changes out to production. Low performers fall behind in every aspect, so the biggest gap has been from between Low and Medium Performers.
The happy trend is that the High Performing group is growing and expanding, so that trend shows that the high performance is attainable for many teams in the industry.
It's not some rare unicorn lands. It's something we can strive for and automation, as we've seen, is a major success factor in achieving that high performance. I've been so lucky to work on high performing teams and I can just tell you, it's so wonderful and enjoyable working with a team that you really trust and you understand the domain and you've got your automation sorted and you can get your continuous delivery going and successful.
We can learn so much now from production use thanks to all the new technology.
That helps us identify gaps in our automated tests and it also helps us to understand what new features or changed features that customers want. And if there are any features they're not using, we could remove those along with our automated tests.
From lean development, we learned about measurements like cycle time —cycle time is from when we start working on a user story, or a piece of a feature, to when we deploy the change code to production.
Rework increases cycle time.
When developers, testers, product owners, and others on the team share the same understanding of the story, when work starts, the cycle time will go down. So that's why practices such as the Three Amigos meetings with a developer, a tester and a product owner, and using structured conversation tools like example mapping to talk about business rules and examples for futures, along with practices like behavior-driven development and acceptance test driven development — those all add value because we understand what we're doing, we automate as we go and so we end up with automated tests that really test what the customers are going to be doing with the feature.
The short cycle time and fast pipelines mean quick feedback on any regression tests failures.
Slicing features up, is starting out with a thin learning release, for minimum viable product, to get feedback from a production use, can tell us if we should keep working on a feature, on what is most valuable to the customers, and it saves us from delivering features that customers don't want.
It can also, like I say, help us identify gaps in our test automation so we can quickly fix those.
I've always heard that hiring good people and letting them do their best work is the key to a successful team.
I like this quote from Liz Fong-Jones: "Cultural and process changes, rather than changes in tooling alone, are necessary for teams to sustainably manage services."
The Harvard Business Review a couple of years ago reported Google's research that psychological safety, the belief you won't be punished for a mistake, is a common characteristic shared by all high performing teams. We need to feel safe to fail.
We need to focus on quality, not on speed. Building those strong foundation of a good test automation strategy, good development practices, good tools, to learn from production, good tools to create our pipeline and get fast feedback — those are the things that let us go faster in the long run, but that happens because we focused on building that quality in.
I learned years ago from Linda Rising about how teams can improve by identifying the biggest problem in their way. Designing small experiments to chip away at that problem, measuring progress, and iterating until we can move on to the next problem. That has really worked well for the teams I've been on.
Frameworks like the Test Suite Canvas that help us ask and answer important questions about our automated test suites, are so valuable. We need to have these conversations and keep improving.
Forming collaborative relationships within your team, and with people outside of your team that your team needs, is key to succeeding with software development.
Katrina Clokie has a lot of great ideas for building bridges to other people in teams in her book.
Retrospectives can sound like a cliché today, but in my experience, if people feel safe to contribute and you focus on chipping away at your biggest current problem, they help you design experiments to make progress. When you want to succeed with your test automation, these are great processes and techniques to rely on.
Thank you for joining me in this learning journey about test automation and DevOps.
I hope you and your team will talk about how you can succeed with continuous delivery or continuous deployment with test automation and keep learning from all these great resources.
Be patient, keep collaborating, and improving one step at a time. Do small experiments.
Enjoy the process of improving your pipelines step-by-step and building your confidence to release small, valuable changes to production frequently at a sustainable pace.
Resources
Agile Testing Condensed, a Brief Introduction, Janet Gregory and Lisa Crispin
The Whole Team Approach to Continuous Testing by Elisabeth Hocke - TAU Course
“Sustainable Operations in Complex Systems with Production Excellence” by Liz Fong-Jones
“High-performing teams need psychological safety. Here’s how to create it” by Laura Delizonna
More Agile Testing: Learning Journeys for the Whole Team by Lisa Crispin and Janet Gregory
