Is That A Tree Sentinel?

Requiring a team support the software and capabilities that they are responsible for is a sound idea.

But the skills and attitude involved in supporting software are often fundamentally different from the skills and attitude required to create it.

And that can cause problems.

Oh God It Is

I've explained what disturbed is a few times already, most recently in this blog post. TL;DR, it's a rotational responsibility where the developers on the team take turns in supporting their users. The role involves answering questions, identifying problems, performing manual operations, and a bunch of other reactive and often time sensitive tasks.

There are a lot of upsides that come from having developers regularly fill the disturbed role, the most important of which is that they get to see their creations being used by real people. It builds empathy for the user and a deep understanding about how the things they build are actually being used.

But the disturbed role comes with some significant downsides as well.

For example, it can be very disruptive. With a large enough team, each developer only ends up doing disturbed once or twice a quarter, but delivering software is hard enough at the best of times. Throwing an extra spanner in the works can make a real difference in the success of both individuals and the team as a whole.

It can also be very stressful. The volume of disturbed requests that a developer has to deal with can vary wildly from week to week, and there are distinct breakpoints where it can become unmanageable for a single person to do it all. The variability can make each disturbed week feel like playing Russian roulette, which adds to the stress.

Finally, it can significantly impact developer satisfaction. Some developers are very good at disturbed, capable of juggling a bunch of different things at the same time without dropping any of them. Some are...not. Given that most developers did not get into engineering in order to do the sort of work that exists in disturbed, it can detract significantly from their general happiness. That, in turn, can create all sorts of knock-on effects around engagement, excitement and motivation, even once the rotation has finished.

In my team, disturbed has been a pain point for a long time. We've made some improvements to the services that we offer our users, which has helped a little bit. We've also split the team into two, equally distributing the disturbed responsibilities between the halves, which helped a lot.

But we can still do more.

It's Coming This Way

In an ideal world, the capabilities that your team exposes to their customers are mature enough that they don't need to be supported. They just work as expected and users are empowered to resolve problems that they encounter. In this world, disturbed does not exist.

In a slightly less than ideal world, capabilities are mature enough that they require minimal support, no longer requiring the full attention of a single person on a week-by-week basis. In this world, disturbed exists, but the impact is minimal.

I don't know if either of those visions are possible to achieve.

Instead, if we accept that disturbed must exist, the problem becomes who should do it.

Enter the Shield Engineer.

Shield Engineers are highly technical people who like doing the sort of work that makes up the majority of disturbed.

Introducing a few dedicated shield engineers into the team to take ownership of disturbed is a win-win situation.

The developers are able to focus on software delivery, while still having a direct channel to their customers as necessary. The shield engineers will keep them apprised of the impact of what they are delivering and have the power to escalate problems as necessary.

Meanwhile, the shield engineers get an opportunity to do what they like doing, fulfilling a more reactive role that is closer to the customer. In addition, they still contribute to the capabilities they are responsible for supporting, so can use their wealth of context and empathy to make things better, or to propose larger projects to make things better.

From a more holistic point of view, the team benefits as well. Having people in roles that they enjoy creates all sorts of positive impacts on motivation and engagement.

From a business point of view, it's not even really an extra cost, because someone was going to have to do disturbed anyway, so it might as well be a person who wants to do it.

I Bet I Can Outrun It...

There are a few things to watch out for though. Specific behaviours that you need to watch for in a world with shield engineers.

The first is throwing things over the fence. You need to ensure that developers are not just tossing "completed" capabilities over the fence to and then moving on to something else. The delivery of changes needs to be more collaborative, with at least some sort of handover, or the shield engineer will lack the context necessary to effectively provide support or feedback on what has changed.

The second is gatekeeping. This is almost the reverse of the first point, where the shield engineers are preventing changes from the developers from being released to customers for some reason. Again, this is solved by close collaboration between the two roles and clear lines of communication, but it's still something to be aware of.

The third is overload. As mentioned previously, the disturbed role can be very stressful, and while shield engineers are better equipped to handle the role, they are not immune to the stresses. Don't just have a single shield engineer. That exposes you to all sorts of risks around redundancy and burnout, and is a bit of a dick move. Have two if you can. Also, make sure the developers understand that they may be called on to do disturbed from time to time, and much be prepared to do so.

The final thing to watch out for is lack of empathy. This goes both ways. The developers need to have empathy for both the customers and for the shield engineers, and the shield engineers need to have empathy for the developers and the customers. Adding a layer of abstraction and clearly defined lines of responsibility can detract from empathy generation across multiple dimensions, so close collaboration and regular positive, respectful communication is critical.

You Have Died

The concept of shield engineers kind of feels like a return to a dedicated operations person, and I'm not sure how I feel about that. It feels a little bit like a movement away from you-build-it-you-run-it.

There are a few teams in Atlassian that have shield engineers already and they seem to be doing pretty well though, so I have faith that it's a workable system. I just haven't lived it myself.

But I will soon.

We've already hired our first shield engineer and they start in October 2023. Next week in fact. We've got another open role which will hopefully be filled soon after.

Within a quarter or two we'll know whether or not it was a good idea.

I suspect I will have less regrets than when I picked a fight with that Tree Sentinel right at the start of my first playthrough of Elden Ring.