How to succeed as a tester when your team is adopting DevOps

Change your mindset

Like agile, DevOps is a set of practices, but it is also a culture, centered around quality.

By deploying small changes to production frequently, we keep risk low and avoid causing customers pain. By using monitoring, observability and analytic technology to “test in production”, we provide an extra safety net to prevent unexpected system behavior.

Culture change is hard, and testers often suffer a lot of stress in a transition to more frequent production deploys.

Management often gets the false perception that adopting DevOps and CD will immediately make everything “go faster”. But it doesn’t make delivering big new features faster – it simply breaks those big features into small slices that are released incrementally and frequently. If the changes are too big, testing is unlikely to keep up.

There’s also the dilemma of how manual testing activities such as exploratory testing can be done if you’re deploying weekly, daily or more often. As testers, we have to make a big mindset shift. There’s not time to find bugs after coding is done. We have to prevent bugs from getting out of development.

That means we have to start testing from the first time a feature idea is discussed, all the way through development. It also means we have to transfer our testing skills to team members who don’t yet have them. My own teams could only succeed with continuous delivery by having everyone test all the time.

For example, I facilitated exploratory testing workshops with developers, and paired with them to learn some techniques. Then they were able to do exploratory testing on each user story as they completed it.

Testers don’t “own” quality – the whole team needs to take responsibility for it.

Build those relationships – ask for help

There are lots of new skills testers need to learn on a team that has embraced DevOps.

Though the idea of instrumenting code to log events in production isn’t new, the technology for it is much improved. We have tools that let us log everything, and analyze all that data to learn how customers use our product and whether it is behaving as expected.

We need to learn how to use these tools. This is pretty intimidating for me, to be honest. Remember, though, we’re a team, we all have the same goal – deliver a great software product that meets our customers’ needs. So, don’t be afraid to ask for help.

In her book A Practical Guide to Testing in DevOps, Katrina Clokie emphasizes the need to build bridges to other people in other roles and teams. We need relationships with operations experts, database experts, product and design experts, customer support, and more.

Asking people with other specialties to help you learn what they do is a great way to make friends as well as to gain knowledge. Invite a configuration expert to come explain the deployment pipelines to your team or your testing community of practice. Ask a developer to pair with you and show how they instrument the code for logging. Invite everyone to a workshop to learn about exploratory testing, and provide cookies or fancy cheese!

It takes a lot of courage to ask for help, especially people you haven’t worked with before. But testing takes a lot of courage, so you’ve got what it takes!

Share the pain, make testing problems a team problem

Here’s a scenario I see all too often: The managers say “OK, we are doing DevOps now”, they put everyone in cross-functional teams, and expect magic to happen.

If you’re a tester who’s stuck with doing manual regression testing, and finding that impossible to do quickly enough for the new pace of deployments to production, please don’t suffer in silence.

Any problem you have, make it a team problem.

I hope that your team is doing frequent retrospectives, and using them to make continual small improvements. This is a great opportunity to get everyone together and ask what level of quality the team is committed to delivering.

Everyone wants good quality, right? Then, bring up what’s in the way of the team feeling confident to continually release small changes to production.

Ask the team to identify the biggest problem and brainstorm experiments to solve it. Form a hypothesis, figure out how to measure progress. “We believe that if we have a tester pair with a developer for 30 minutes of exploratory testing on each story, we will see a 20% reduction in bugs found after the story is delivered.”

If the experiment doesn’t work, try something else – it is all learning, it is all progress.

Ask everyone to share the pain of manual regression testing – that is great motivation for developers to write more testable code. When developers take responsibility for writing and maintaining automated regression tests, with testers helping them to specify the test cases, the team is much more likely to perform well in a continuous environment. And watch for the point of diminishing returns.

My last team had thousands of automated regression tests, but we still had a checklist of tests that couldn’t be automated. Trying to get through that for twice-weekly deploys meant not enough time for exploratory testing – and serious bugs were getting out to production.

We simply stopped doing that small set of manual tests, and worked on our ability to quickly identify and fix problems in production.

Make quality and testing visible

Making quality visible is a big challenge. How do you measure it?

Your delivery team can set quality goals and find metrics to show progress towards those goals.

Ask customer support and sales for customer happiness measurements. Make those metrics visible.

Here’s an example. The team is struggling with failing automated test suites, and one problem is some tests are “flaky”, they fail sporadically even though there’s no bug in the code. You’re spending a lot of time analyzing failures to see if it really was a regression or just an issue with the test itself.

Make the frequency of failures visible on a big monitor. Set a goal and budget time to fix or remove some number of flaky tests per week. If the failure frequency doesn’t go down, think of another experiment to try.

My previous team had bi-weekly “share and tell” meetings where each sub-team or group got a few minutes to show what they were working on. This is a great opportunity to make everyone aware of a new technique you’re trying, such as a framework to build shared understanding before starting work on a story.

Whatever big problem is impeding successful continuous delivery, make it visible with a big wall monitor or chart or its virtual equivalent. A couple of my teams have resorted to a flashing police light to make failing builds or production problems more visible so that someone would take responsibility to investigate and make sure the issue got fixed.

Brainstorm together to find ways to make failures AND successes visible! Be sure to celebrate every success, no matter how small.

Make use of online resources, get learning buddies

We are so fortunate these days to have so many learning opportunities: online courses and webinars that are often free, articles and blog posts, meetups and conferences, online communities such as Slack workspaces.

There’s a wealth of information out there about DevOps, continuous delivery and deployment, test automation, logging, monitoring, observability, and more.

If you are a manager, make sure your team members have plenty of time for learning. An hour of learning per day would quickly make any of us able to contribute more value for our team. It’s an investment.

If your situation allows you to take time for learning outside of work hours, that will pay off in many ways.

Power up your learning by getting one or more friends to learn with you. That way, if you get stuck, your learning partner may be able to help you move forward. Plus, it’s more fun to learn together.

Technology is going to keep surging forward. We can’t keep up with all of it, but find the areas that give you the most joy or satisfaction, and keep learning. That’s going to make you successful, whatever you decide to try!