Growing a platform: Introducing API versioning in Intercom
We believe that shipping is our company’s heartbeat and that we should ship frequently with small incremental changes. This principle enables us to get feedback and iterate quickly.
However, until recently we couldn’t apply the same philosophy to our APIs. We weren’t able to fix bugs, improve the functionality of our APIs or even keep up with our constantly growing products. That’s because API interfaces are like contracts, which can’t be easily changed once released to the world.
Developers building on the Intercom platform rely on the current existing functionality of our APIs, and potentially backwards incompatible changes might break the flows in which they are used. Any changes generally require advance notice to developers to let them know that their app will be affected and that they should migrate to the new functionality. Forcing an update also means that developers have to allocate the needed resources to update their apps, which can be quite time consuming depending on the changes.
“API interfaces are like contracts, which can’t be easily changed once released to the world”
Even with enough notice, rolling out changes that break things is not straightforward – coordinating our release with our developers as they update their apps has to happen in a way that we avoid intermediate states where the functionality is broken. If there’s a problem, rolling back to the old state isn’t always easy. This ends up being a no-go for us in most cases because it’s not a reliable, safe solution.
We wanted to fix these issues and worked on introducing versioning for our REST and Messenger Framework APIs. The enhanced creation and management of multiple releases of our API would not only improve our APIs as a platform but also improve the developer experience. We focused on providing a solution that reduces friction for developers by allowing them to upgrade their apps when they wish in a safe way.
Listening to our customers’ needs
We wanted to identify the main friction points when developing apps and updating to the latest version of our API in order to minimize the complexity and the risk of the upgrades. So, we conducted research sessions with our users to understand how they use our platform APIs as well as other platforms and the main problems that they face.
It became clear that developers need to be able to assess the changes in a single place with a very clear, in-depth changelog. They needed any changes to our API to fit their processes. We needed to allow them to plan upgrades with plenty of notice and provide them with very convenient and flexible ways to test their code before performing an upgrade, as well as enable them to roll back quickly if anything went wrong.
Our approach to API versioning
We have our own open source library, called Requisite, so we can strongly define request and response models for serialization in an elegant way. We use this library to implement our models. The simplified version of our UserResponse serializer, which transforms the internal user model to the API payload, looks like this:
class UserResponse
serialized_attributes do
attribute! :id
attribute :user_id
attribute :email
attribute :phone
attribute :name
end
End
Introducing version changes
Inspired by the Stripe API versioning blog post, we introduced the concept of a version change, which represents a single change made on an API model. It consists of a description of the change, the public model it’s targeting, and a transformation to the previous version of the model. Multiple version changes are compiled into a version.
Our controllers always operate on the latest version of the models. If the API receives a request for an older version, we transform it into the requested version using the version change. We use the same transformations to parse request payloads from older versions to the latest one, with the only difference that we apply them in an ascending order.
How a single version change works
If we want to rename the user_id on the User public model to customer_id, we would need to update the UserResponse model to use the new attribute and create a new VersionChange that transforms the payload back to the previous version.
class UserResponse
serialized_attributes do
attribute! :id
attribute :customer_id
attribute :email
attribute :phone
attribute :name
end
End
class RenameUserResponseCustomerId < VersionChange
description "User model will now use customer_id instead of user_id"
transform UserResponse do |data|
data.merge(user_id: data[:customer_id])
end
end
The idea here is that when we need to make a change, we always update our public models to the latest version and then use a VersionChange subclass to roll back that change and get the previous version of the public model. We then group multiple VersionChanges in a Version.
VERSIONS = [
Version.new(
id: "1.0",
changes: [
InitialVersion,
],
),
Version.new(
id: "1.1",
changes: [
RenameUserResponseCustomerId,
AllowSearchMultipleUsersByEmail,
],
),
]
How we chain multiple version changes
It’s possible to chain many VersionChanges for a public model and apply multiple transformations. This can happen if we have multiple VersionChanges on a single model in multiple versions.
To expand on the previous example, if we decide to rename customer_id to person_id in version 1.2, we should update the UserResponse to use the new attribute, and add a VersionChange that transforms the payload from v1.2 format to the one expected by v1.1:
class UserResponse
serialized_attributes do
attribute! :id
attribute :person_id
attribute :email
attribute :phone
attribute :name
end
End
class RenameUserResponsePersonId < VersionChange
description "User model will now use person_id instead of customer_id"
transform UserResponse do |data|
data.merge(customer_id: data[:person_id])
end
end
When a v1.0 request comes in, we first apply the RenameUserResponsePersonId transformation, which transforms it to v1.1. Then, we pass it to the RenameUserResponseCustomerId transformation, which transforms it into v1.0. This way, any response payload can be transformed into a version that the client expects.
With our controllers only operating on the latest version, all the code related to previous versions is contained inside the VersionChanges objects. This design allows us to have a single implementation of the logic, eliminating duplicated code and reducing the extra overhead in the maintenance, making our codebase more robust, reliable and maintainable.
How API versioning works in practice
Adding the versioning allows us to implement feature requests and bug fixes that highly benefit a specific audience of developers without forcing everyone to upgrade. A clear example of this is searching by email in the Users endpoint of our REST API.
In version 1.0, we can only return a single user when searching by email. If multiple users are associated with the same email address that endpoint returns an error. Even though this is not a desirable behavior, and the solution is simple to implement, some developers depend on it. A fix could break their apps. Their code expects response payloads in a specific format and will not be able to parse a list as a result.
In version 1.1, we now support fetching multiple users with the same email. Every developer can opt-in to using this version when their code is ready to support it.
“We can now release changes to all our platform features more frequently and safely”
The design we followed makes our platform versions flexible and easy to test. We allow testing isolated requests using a specific header to ensure that the app works well with the new functionality before updating its version. When everything is tested, updating through the Developer Hub will make all the requests and responses coming from this app to be interpreted in the new version.
Rolling back in case of a problem is also immediate through the same process. We can now release changes to all our platform features, like REST API, Messenger Framework and webhooks, more frequently and safely. This allows us to improve the quality of our platform and therefore the apps built on top of it.
Shipping fast and growing a platform
Developing APIs to support fast growing products comes with a lot of challenges. This felt like a particular constraint at a company where shipping software fast and often is integral to how we work and build products. We wanted to enable Intercom engineers to support our APIs better. By implementing versioning, we’re now able to align ourselves with the rest of our engineering team and to ship continuously.
Ultimately, if the old version of our APIs was like a contract that required careful renegotiation every time we wanted to make an improvement, implementing versioning frees us from that – now we can improve and iterate on our APIs much more dynamically, with all parties able to feel secure and confident in the process.
Additional contributor: Gabriel Anca Corral works as a product engineer at Intercom.
