Striving to make Intercom delightfully fast

Many of our customers spend their entire working day in Intercom, switching between the inbox, user list, and user profile pages. We aim to delight these folks with a beautifully fast product, not to frustrate them with slow page renders or web browser memory leaks.

At various times we have made significant improvements to Intercom’s performance, but we haven’t always hit the bar we set ourselves of ensuring performance is consistently excellent.

This blog post is the first in a series of regular performance reports. We’re going to share raw performance numbers, highlight positive and negative changes, and discuss upcoming projects that will have an impact on performance.

We are taking inspiration from great companies like Etsy, that have tackled these problems before. Sharing performance details regularly will:

• Provide our customers with visibility in to our performance numbers
• Allow us to celebrate performance improvements
• Keep us honest by making it clear when performance worsens

Recent performance improvements

First load:

We’ve spoken previously about moving the front end of Intercom to Ember.js, and how it allowed us to deliver a richer, more delightful experience to our customers.

When a customer loads Intercom, the first page their browser loads is a simple html page that boots the Ember app. In order to speed up the boot process, we began embedding some JSON in this page. But as time went on, increasingly more data was embedded, which actually resulted in slower performance. We were seeing median response times of ~700ms. In July we shipped some optimisations that dropped the median down to ~350ms:

That was a good result, but the p99 (slowest 1% of all requests to this endpoint) was still unacceptably slow at almost 3 seconds. Last week we improved these worst case scenario requests, and they now take around a second:

User list:

The first page our customers see when they load Intercom is the user list. It allows them to see who their customers are, and what they are doing. The user list was the first part of Intercom that used Ember.js. When we switched it felt lightning fast compared to the previous version. However we did not have any client side instrumentation, and as we added new features (and increased complexity), performance degraded significantly. When we started recording client-side performance metrics, we discovered a median of 1500ms for simply rendering a page of 30 users. It was frustrating to realise that we had allowed the customer experience to degrade so badly.

In July we dramatically simplified the process of rendering each row in the user list, and reduced the median time to render a page from 1500ms to 600ms. We’re happy with our progress, but it’s still not fast enough.

Inbox:

Heavy Intercom users tend to spend a lot of time in the inbox. We added many new features this year (Real-time Chat, tooltips, hover cards, animations), which had a negative impact on performance. Long conversations (with hundreds of comments) were particularly badly affected.

We recently spent a lot of time working on inbox performance, and have made a significant impact.

Here’s what we did:

• Started paginating conversations. Rendering the entire conversation history meant that as conversations got longer (common with real-time messaging), the time taken to render them increased linearly. Paginating conversations means that we only ever render 25 parts at a time – leading to a much lower and more consistent render time and a better customer experience.
• Avoided unnecessary work. Previously we re-rendered the entire conversation history every time somebody replied. Now we render just the latest comment, which makes things much snappier.
• Simplified client side rendering very significantly, which halved the rendering times of individual comment components without removing any functionality.
• Implemented client side caching of conversations, which greatly speeds up switching between conversations.
• Numerous server side controller improvements. Here’s an example of the impact on the  controller that returns saved replies:

Likely Upcoming Projects

• Further improve the worst case scenario (p99) performance on the page that boots the Ember app by minimising expensive database calls.
• Continue to improve the customer experience on the user list by further reducing server and client side latencies.
• Use a combination of alarming and regular performance metric reviews to ensure that we do not slip back on any of the recent performance improvements. We intend to continually ratchet down on performance until we achieve an experience that we are proud of.

Target: Delightfully Fast

Moving the front end of Intercom to Ember.js allowed us to deliver a far richer experience to customers. It also delivered snappier performance, as state changes tend to be small operations that don’t require re-rendering the entire page. Our failure to adequately instrument the client side application meant that as we added more features, previously quick operations started to lag, and in some cases we ended up with worse performance than we had before.

We now collect, monitor, and act on client side metrics, and while we are nowhere near satisfied with our current performance numbers, we have made significant improvements over the last quarter. We’re determined to make Intercom delightfully fast for our customers, and are looking forward to sharing the next performance report with you.