The importance of process: How we scale sustainable engineering processes at Intercom

Fast-growing companies are in a constant state of flux. With new learnings and challenges emerging almost daily, it may feel difficult to justify spending valuable time crafting principles and processes rather than features and products.

There are, however, real benefits to defining and refining processes to enhance team performance and alignment. This post is an adaptation of a talk I recently gave at the LeadDev Together conference about how Intercom sustainably scales its processes to sync teams, streamline workflows, and increase efficiency.

Processes create clarity and structure

An organization is a collection of moving parts, consisting of people and resources that work towards a common goal. It’s critical that they align at a high level to a clear mission, vision, and strategy – without this, collaboration and productivity issues can quickly arise, affecting your ability to deliver on your objective.

But as companies grow, the connections and dependencies between their various parts can become increasingly complicated, and make it more challenging to operate in a co-ordinated and effective way.

In his 1975 book, The Mythical Man-Month, Fred Brooks observed how adding people to a team often drove inefficiency, increased communication overhead, and had little impact on the efficacy of the team. According to Brooks, adding more people to a late software project usually delays it even more. Simply increasing the number of people at your company or working long hours to offset inefficiencies doesn’t guarantee a return to effective, co-ordinated work. It can actually make things worse by creating more communication lines, which can increase the possibility of misalignment.

Brooks’s Law: increasing the number of people leads to “combinatorial explosion”, with a rise in communication overhead

Transparency issues can also quickly arise as you scale. Decisions can occur “in the dark” in one part of the organization that can have unintended consequences for other parts of the business. For example, setting incorrect expectations with a critical customer might force a whole team to change their priorities, which could cause a significant engagement problem for your team. The result? You may see increased attrition rates as disempowered teams scramble to adapt, and you could negatively impact revenue.

It’s critical that you regulate how you interact and make decisions as a business. To do this, you need to continuously evaluate and improve any dysfunctional parts within your operations. As you scale, it can be difficult to maintain a “big picture” view of how your actions impact other areas of the business. Add regular, rapid decision-making to the mix, and you can see how multiple people making independent decisions can threaten overall alignment and your ability to make progress.

“Principles and processes provide us with a systemic and scalable way of repeating the behaviors that we strongly believe will lead to positive outcomes, and help us to avoid repeating past mistakes”

So, how can you restore order to chaos?

Defining principles and implementing processes to actively manage the alignment that came so effortlessly when the company was small and the demands were less intense allows you to codify your successes and lessons learnt.

At Intercom, our principles and processes provide us with a systemic and scalable way of repeating the behaviors that we strongly believe will lead to positive outcomes, and help us to avoid repeating past mistakes.

Processes help scale an engineering organization

As an engineering leader, what are you ultimately responsible for? For me and my peers, it’s building Intercom. Our work focuses on:

• Supporting and improving the health of our existing revenue lines by protecting our availability and costs, addressing quality issues within existing products, and mitigating security risks.
• Adding new revenue lines by building new products or extending functionality.
• Protecting our ability to move fast. Put simply, this means improving on our ability to achieve my previous two points. We believe that speed is a competitive advantage in the market that we operate in and is conducive to success. The sooner we get our work in our customers’ hands, the sooner we’ll solve their problems, and the faster we’ll grow.

Based on my experience, there have been three key types of processes that have consistently helped us to scale and deliver on our goals. These are:

1. Processes that improve the quality of our execution and manage how we build, support, and evolve systems to our expected standard.
2. Processes that help support our people and aim to increase their engagement and growth. These processes are often neglected by smaller companies and need to be implemented early as they can be messy, costly, and unethical if mismanaged.
3. Processes that help align the business and your ability to deliver common goals like a desired customer outcome, revenue goals, or business sustainability targets.

While all these might seem like they address different issues, don’t be fooled – this is a deeply interconnected system, and all parts can unintentionally affect other seemingly unrelated parts of the system. For example, consider a poorly designed application that causes a lot of out-of-hours incidents. This type of disruption left unchallenged can quickly increase customer churn, burn out your employees, and derail revenue goals. So you can see how a single weak process can create a ripple effect that impacts the performance of the whole business.

The processes you decide to prioritize will depend on many factors. There’s no perfect formula that fits all organizations – a fledgling B2C startup will have vastly different needs compared to a large well-established fintech company, and their processes should reflect this difference. What you optimize for is hugely dependent on your goals, constraints, and ultimately, the kind of organizational values you want to uphold.

“Having good intentions only takes you so far! The consequences of poor decisions are unfortunately real. While we can all make bad calls, it’s our responsibility to not be negligent”

But as many of us have learned this year, the context of a system can quickly change. Constraints can appear at any moment due to a change of strategy, regulation changes – or a pandemic! – to suddenly render your processes ineffective.

Thinking more holistically about the context those processes operate within is critical to their success. As the context changes, you should reconsider your approach and any side effects a change can cause. Having good intentions only takes you so far – the consequences of bad planning are, unfortunately, real. And while we can all make bad calls, it’s our responsibility not to be negligent.

So, before you lift a process from a different company or try to implement something that has worked for you in the past, make sure you fully understand the goals, constraints, values, and principles you are optimizing for and their compatibility with the solution you are proposing at this specific point in time.

Successfully rolling out effective processes

With a good understanding of the priorities, values, and principles of a business, you are better equipped to identify any issues that need to be addressed. These could be highlighted by customer complaints, frequently missed targets, or dissatisfaction among staff noted in an internal survey.

Implementation of an effective process in your organization can fail because:

• People don’t follow it. There is no accountability for its implementation. Some people might not even know that the process exists, while others may find it unnecessary and believe it impedes their ability to do their job.
• People follow it but are frustrated and resent the process. The process may not resolve a problem they care about. For others, it may resolve the issue in a different way to how they would like to see it fixed; and so the perceived cost of the compromise outweighs the benefit of following the process.
• It’s a bad solution to your problem or it’s unnecessarily prescriptive. This is particularly troublesome as it can stifle much-needed innovation and autonomy among teams (yikes!), leading to unchallenged and blindly followed assumptions. Even worse, this particular barricade can inadvertently encode people’s biases into established ways of working, which can create systemic issues.

So, how do you effectively define a process, and even more importantly, achieve buy-in from stakeholders?

How to implement a new process

Here are some steps I recommend for effectively defining and implementing a process:

1. Clarify and agree on the problem you are trying to solve and if the problem is worth solving now – identifying if it’s a team problem or a company-wide issue will help you define the scope of the solution.
2. Identify owners and understand who will be impacted by the changes – the DACI decision-making framework can be helpful here. It’s important to include a diverse representation of voices in this group who will potentially be impacted by changes. In fact, one of our principles at intercom is that, “Those closer to the problem should be empowered to solve or own the problem.”
3. Agree on the goals and principles you would like to achieve with those involved and be explicit about any constraints or assumptions that you may have.
4. Map some options and don’t restrain your ideas during the ideation phase. Once you have some “big picture” ideas, scale them down to suit the scope of the problem. Small, iterative changes that substantially improve a system can often trump a more thorough but complex idea that strains resources and doesn’t quickly produce results.
5. Get extra feedback at this point from a representative sample of those impacted by the changes. It’s a chance to add more diverse and potentially critical voices to the mix so you can test your assumptions for accuracy. Once you’ve assessed this feedback, you can either evolve your approach or proactively address any concerns in later communications.
6. Test your solution prior to a full rollout. It’s always good to enlist the help of a few willing participants to validate the solution, particularly on ease of use. It’s similar to slowly rolling out a new feature to your customers to identify performance, usability, or uptake issues.
7. Finally, communicate the change appropriately. Ideally, you’ve been providing regular progress updates to those that need to be informed. But you’ll need to create good educational materials to empower those affected by these changes to engage with the process and achieve the desired results. This is a crucial step that is often neglected. Providing up-to-date documentation is essential to reinforce the why of the process and set clear expectations with those who need to follow it.

Unfortunately, it’s not enough to merely roll out a process. If you want it to produce tangible results, you’re going to have to establish controls and embed accountability into its execution. Processes can be easily misused or abandoned in a rapidly evolving business, so it’s important to have a way to assess their effectiveness, benchmark success, and quickly learn from your mistakes. Always be open to evolving or removing a process entirely if it’s not needed or fit for purpose.

What this means for engineering leaders

It doesn’t matter if you are designing new processes, modifying existing processes, or ensuring your team is compliant with established processes in your organization. We are – in all cases – responsible for how successfully our teams engage with the system, and how it affects them and their work.

Remember, our responsibility as leaders is to meaningfully contribute to the success of the business through tangible results and highly engaged teams. As you begin to scale and create an operating system for your organization, be mindful of the impact seemingly inconsequential decisions and processes can have on both business outcomes and individuals.