Questioning tech: The past, present, and future of survey tools
Surveys aren’t the sexiest of topics. But they’ve come a long way since the first census was taken, and, with a slight nudge from recent technology, they’ve got plenty more to go.
Last week, we introduced three brand-new products to help businesses drive customer engagement and growth. One of them was Intercom Surveys – a tool that connects you to valuable customer insights and enables you to turn those insights into action in real-time. Intercom Surveys was, curiously enough, the result of survey, one we had sent to our users to gather their feedback on one of our features.
This got us thinking about the nature of surveys. Not just online surveys, but all kinds of surveys, everywhere. And so, we thought we’d take you on a quick tour into the beginnings of surveys and tell its story – or rather, an abridged version – throughout the times, from the first mention in the Book of Exodus and Chinese mythology many centuries ago to the effortless, ubiquitous version we have today.
Today, you’ll hear from:
- Zoe Sinnott, the Senior Product Manager at Intercom and one of the people behind Intercom Surveys;
- Andrew Whitby: Data Scientist, Economist, and Author of The Sum of the People: How the Census Has Shaped Nations, from the Ancient World to the Modern Age;
- Rick Klau, California’s Chief Technology Innovation Officer;
- Tristram Hooley, Researcher and Professor at Inland Norway University and co-Author of the book, What is Online Research?
Of course, this would hardly be Intercom if we didn’t take a moment to look ahead at the future of online surveys and how Intercom Surveys can play a part in that. And if you want to find out more about it, you can read our latest article or visit its own feature page.
If you enjoy our discussion, check out more episodes of our podcast. You can follow on Apple Podcasts, Spotify, YouTube or grab the RSS feed in your player of choice. What follows is a lightly edited transcript of the episode.
Humble beginnings
Liam: Hello and welcome to Inside Intercom. I’m Liam Geraghty. If you were at our recent virtual event – New at Intercom – you will have heard about some of the amazing new products we have just launched. One of those products is Intercom Surveys, which allows companies to ask questions from right within their product, at exactly the right time and place, and then take direct action with the answers and insights they get. That’s because Intercom is a single connected platform, so the survey results can trigger powerful workflows for everyone from the sales team, to the marketing team, to the support team.
It’s so innovative, and it got me thinking about the path that led to this point. The history of surveys is essentially the history of the technology used to collect and interpret information. As tech has developed, so has our capacity to know ourselves at a large scale. We’ve all experienced this tech at some stage in our lives. Take Zoe Sinnott. Zoe is a senior product manager at Intercom and one of the team behind Intercom Surveys.
Zoe Sinnott: I remember my parents filling out the paper census, and I remember being really curious about what it was and why we had to do it. The last time I filled out a census, a few years ago, I was living in Canada and I was really pleased to see they’d moved the census to be online. That was real progress. But what made me laugh was that they still sent someone around to knock on your door to make sure you filled it out. So, yeah, it hasn’t changed too much.
“There was a point in time when data on the scale of megabytes was hard to come by. And the census was one of the sources of material”
Liam: It’s true, someone literally called my door the other day to drop in the form for Ireland’s 2022 census. And actually, that’s a pretty good place to start for today’s episode – a deep dive into this world. The story of the technology behind asking questions and discovering more about people, at scale.
Liam Geraghty: Okay. Let’s see. Census of Population of Ireland. Name, Liam Geraghty. What is your place of birth? Dublin. I’m filling out the census form. It’s been 6 years since the last one. The first-ever Irish census was taken in 1821 after an unsuccessful attempt in 1813. And while filling in all my answers marking each little box, it strikes me that it’s kind of an amazing system that in one shape or another has been around for thousands of years, all over the world.
Asking questions leads to knowledge, and the world has always sought to find out more. And not just about our population size or place of birth, but about every aspect of our lives. Surveys have evolved to allow everyone, from your local pet store to huge multinational corporations, to ask for our opinions on their products and services. This curiosity and passion to push forward using surveys started with the census. In a way, it was the original big data.
Andrew Whitby: It’s kind of wild to think about it, but there was a point in time when data on the scale of megabytes was hard to come by. And the census was one of the sources of material. Censuses and tax records were the kind of things governments kept track of at the level of millions of people, and practically nobody else in society kept observations on that many things. So really, at least for perhaps a hundred years, the technology of data and census-taking really grew hand in hand.
Liam Geraghty: That’s Andrew Whitby, author of The Sum of the People: How the Census Has Shaped Nations, from the Ancient World to the Modern Age.
“You find these things dating many thousands of years ago”
Andrew Whitby: It’s difficult to pin down the precise earliest example in part because censuses seem to be so old. They pop up in mythologies of different cultures and in times where we really can’t be very precise about dating or even about whether they existed at all. But you can find censuses in the Old Testament of the Bible. The first kind of mention of a census is in the book of Exodus – there’s a description of a census-taking procedure, which is kind of a tax collection, like a lot of these early censuses were, of collecting half a shackle from each person, with that money going towards the construction of the tabernacle, a kind of temple in the desert.
In Judeo-Christian tradition, you have that early census-taking, but you find the same thing in, for instance, Chinese mythology. You’ll find a census sent similarly from 2000 years before the common era – the census of Yu the Great, who was a famous mythological emperor of China responsible for a lot of the flood mitigation efforts that made much of the terrain of China a livable, viable agricultural area. He was said to have taken the first census of China. So, you find these things dating many thousands of years ago. And I think we can assume that whether or not those events happened, there was some sort of census-taking going on at the time.
Liam Geraghty: The word census is a Roman word. It comes from the Latin, censere, to assess.
Andrew Whitby: And that indicates there is this strong connection to the Romans. The Romans had a kind of census procedure throughout the Republican period and then into the empire where they would use the census to structure their society. It was very different from the census of today, which is an anonymous exercise. Today, you fill out a form, it goes into some kind of statistical aggregate, and then those aggregates are published back out. There’s no individual information. The Roman census was very different from that. You stood before these important officials of the Roman system and you would declare yourself: you would say who you were, what your age was, who your wife was, how many children you had, what kind of financial income and lands you might have… And that was used to establish the strata of Roman society. The Roman census is one of the most famous historical census-taking systems, and that’s where we get the modern word from.
Liam Geraghty: Censuses were taken on everything, from bundles of knotted thread to wax tablets, and they were engraved on stone if they were to be recorded, but eventually, a technology would arrive that would transform communication and record-keeping – paper.
“The number of census questions becomes dramatically larger, and at some point, you reach this crisis point where you need to have a way of literally, physically manipulating this paper”
Andrew Whitby: The problem with paper is it becomes kind of large and unwieldy. In the United States, there was this period when the population had grown very dramatically, from maybe 4 million in the first 1790 census to many multiples of that afterward. Plus, the number of questions becomes a lot larger – the initial US census was just asking for basic characteristics such as sex, age, and race. But later on, they started asking a lot more questions about occupations, disability, education, and things like these. The number of census questions becomes dramatically larger, and at some point, you reach this crisis point where you need to have a way of literally, physically manipulating this paper.
An increasing gap
Liam Geraghty: Think about that. Millions of entries, no spreadsheet software. Census officials struggled to quickly and accurately process the information they’d collected. That is, until Chief Clerk of the Census, Charles Seaton invented the Seaton Device. It was a breakthrough in how all this data could be interpreted.
So, imagine you work at the census office. You have this questionnaire on a large, wide sheet of paper, which looks similar to a spreadsheet you’d see nowadays, but you want to compare some of the columns, such as results for men and women. Because it’s this big sheet of paper, they’re spread out, they’re not side by side, so it’s just awkward. This is where the Seaton Device comes in.
“The census had an increasingly important role, not least of which was to determine how the political representation would be assigned”
Andrew Whitby: The Seaton Device had this complicated system of rollers where you could take this wide sheet and roll it through these rollers so that two columns would appear side by side. To me, it’s the exact functionality of the hide columns feature in Microsoft Excel, Google Sheets, or a modern spreadsheet, but it’s being done physically and mechanically with paper. That was one of the ways these census officials developed to try and deal with this massive paper that had become really difficult to work with.
Liam Geraghty: As tabulating machines evolved, so did America. The United States Census of 1880 came at a crucial time in US history.
Rick Klau: The country was going through this massive transition from a mostly rural country to increasingly concentrated in urban locations. The population was growing.
Liam Geraghty: That’s Rick Klau.
Rick Klau: Currently the Chief Technology Innovation Officer for the state of California.
Liam Geraghty: And Rick says the country was coming into its own as an industrial power.
Rick Klau: The census had an increasingly important role, not least of which was to determine how the political representation would be assigned. Of course, it’s dependent on population, where those people are, how many people there are. And the census was the only way the government had to know what those numbers were and how to then make the political allocations that would come from it.
“The 1880 census didn’t finish getting calculated until eight years later”
Liam Geraghty: The problem with the growing country was that the census was also growing. The government was collecting more data than it could tabulate.
Rick Klau: The 1880 census didn’t finish getting calculated until eight years later.
Liam Geraghty: Or, to put it another way, only two years before the next census was due to start.
Rick Klau: Well, and if you went into that decade wanting to know how we were going to allocate political representation, how we were going to assign taxes and figure out all of these necessary dependencies that we take for granted today, then they had to wait for eight years before they even had the raw data that would allow them to act on it. Which, of course, isn’t all that convenient.
Liam Geraghty: This is where a guy named Herman Hollerith comes in.
Rick Klau: He was a clerk in the census office in 1880, though he didn’t stick around for the conclusion of the census.
Liam Geraghty: Who can blame him?
Rick Klau: He was a pretty precocious guy, having completed college in his mid-teens. I think he was 18 or 19 years old when he had a job in the census office and got to see firsthand just how broken the process of collecting, analyzing, and reporting on that data was. Shortly after being frustrated by that job at the census, he left to become a mechanical engineering professor at MIT at the age of 22. So, he was not a particularly patient individual, as far as we can tell.
Liam Geraghty: Which probably helped get Hollerith thinking about a machine that could speed up the process. He just needed some inspiration.
The Hollerith machine
Rick Klau: A fascinating coincidence, he had a brother-in-law, I think it was, who was active in the textile business. At some point, Herman and this brother-in-law shared an apartment or a house, and the brother-in-law was using something that had existed for decades in the textile business called the Jacquard loom, which was essentially using punch cards to store very complex designs for textiles. This notion of using punch cards for complex processes was not new to Herman Hollerith and, at this point, was nearly a century old in the textile business.
Liam Geraghty: Regardless, it was exactly the inspiration Hollerith needed.
Rick Klau: They were storing data. And what Hollerith figured out was that if there was stored data on the punch cards, he could build a machine that could count what was stored and store the derivation of those calculations. If you think about the purpose of a census, you’re capturing a lot of stored data about how many people there are, what their demographics are, where they live. He was then able to build a machine that counted the stored data and did so, as you might imagine, at a far greater speed and with far higher accuracy than they’d been able to do before.
Liam Geraghty: Hollerith set to work on building his machine.
Rick Klau: I think necessity was the mother of invention quite literally here.
“They put Hollerith’s machine to work on the 1890 census. The job was done in just two years, saving the government $5 million”
Liam Geraghty: The census office held a contest. They said, “If anyone wants to try and help us solve this problem, we’ll hire you.”
Rick Klau: Only three people entered the contest. The third-place submission tabulated the data the contest had set in a little over 55 hours. Hollerith’s machine did that 10 times faster, in just over five hours.
Liam Geraghty: They put Hollerith’s machine to work on the 1890 census. The job was done in just two years, saving the government $5 million.
Rick Klau: The US was not the only country that needed to survey its population. He ends up starting a company.
Liam Geraghty: People began to see the potential for this, not just for censuses around the world, but for businesses – because the data on the cards didn’t have to be about population size. They could be about a customer or a product.
“In the past, surveys were things others did to you. With the internet, it became possible for anyone to create a survey and have the data captured in a way that made it easier to summarize and synthesize”
Rick Klau: He then finds adjacent businesses for the storage and tabulation of large data sets for insurance companies, for railroads, for post offices… and he ends up doing business all over the globe. I think this idea that machines can do things far faster than we can would look very familiar to him. In the past, surveys were things others did to you. You were asked to fill out surveys. With the internet, it became possible for anyone to create a survey and have the data captured in a way that made it easier to summarize and synthesize.
Liam Geraghty: Okay. This is where we have to zip through history: 1947 – It’s now possible to use keyboards to input data instead of punch cards; 1963 – the mouse, as we know it today, is created; 1975 – the first portable computer; 1986 – more than 30 million computers are in use in the United States; 1991 – the World Wide web.
Connected world
Liam Geraghty: The World Wide Web was fast. Well, you know what I mean. For the first time, surveys could reach the entire population, not just of a country, but the world.
Tristram Hooley: Surveys that you used to get in the early days of the web in the early 1990s, I remember what they looked like – lots of circles with buttons and boxes in them.
Liam Geraghty: That’s Tristram Hooley. Tristram is a professor at Inland Norway University. In the past, he spent a number of years running online surveys for businesses and he co-wrote the book, What is Online Research, in which the authors pinpoint the birth of the online survey.
Tristram Hooley: We reckon that the first online survey was in 1986. It was actually a survey about surveys, which is always a nice thing. They were basically just figuring out whether it would work. That was before the World Wide Web. The Internet had existed in various forms since about the 1960s, and so, by the 1980s, people were starting to experiment with this.
“Over about 20 years, it goes from being a really wild and cutting edge thing to being something that almost every business has experimented with in some way”
Liam Geraghty: Similar to how Hollerith’s tabulating machines were mostly used by governments first, the internet was largely used only by academics in scientific disciplines and the military in the 70s and 80s.
Tristram Hooley: What happens from the 1990s is that we get this sense that the general population is on it. And from the point my point of view, that’s when it really gets quite interesting because you start to be able to access some kind of subset of the general population through online surveys. It becomes easier and easier. You don’t have to be a technical genius to start doing this stuff, and so you start to see more and more people doing it. Once you get into the 2000s, you’ve got things like Web 2.0 Technologies, and as part of that, you get things like SurveyMonkey and so on coming along. Then, it becomes a complete ubiquitous plague of everybody doing it. So yeah, over about 20 years, it goes from being a really wild and cutting edge thing to being something that almost every business has experimented with in some way.
Liam Geraghty: The web really did transform surveys. Before then, businesses took the traditional routes to find out what their customers thought.
Tristram Hooley: You’ve always been able to get a clipboard and go and stand out on the street and ask people questions. You’ve always been able to mail people surveys, and for at least a hundred years or so, you’ve always been able to call them up on the telephone and ask them questions.
Liam Geraghty: Those things just became so much easier with the arrival of the Internet.
Tristram Hooley: It enables you to access a community that’s much wider than the community you might actually geographically be located in. It allows you to reach out to people who perhaps might not be just walking along the street or might not respond to phone calls. It might allow you, for example, to survey populations like prisoners or disabled people who wouldn’t be able to access through ordinary forms of surveying. So, it’s got a number of advantages. It opens up new possibilities. It also probably makes it a lot cheaper, although there are obviously still costs to it.
If I really love a particular brand or a particular company, I might want to have some influence on how they understand their customers, what they might want to produce in the future, how they might want to change their service, and so on. And so, there are those things about your affiliation and affinity with the people who are asking you the questions, and you can hack that a little bit by personalizing things in various ways that we can do online.
“If you want to claim some sort of representativeness of what you are doing, it’s not just getting people to fill it in – it’s getting the right people to fill it in”
Liam Geraghty: Just like when the American census had problems back in Hollerith’s day, bringing surveys online comes with its own unique challenges.
Tristram Hooley: The biggest one is getting people to fill them in. And obviously, in the current environment where there are literally millions of online surveys being sent to everybody every day, many of them of questionable probity, really, people aren’t massively inclined to fill it in. You’ve got to get hold of a decent list of people – throwing it out on social media probably won’t generate all that much. It does depend on who you are and what your brand is and so on, but how do you get people to fill it in?
And then, obviously, depending on what you are doing this for, if you want to claim some sort of representativeness of what you are doing, it’s not just getting people to fill it in – it’s getting the right people or the right spread of people to fill it in. And it’s probably more difficult to do that with an online survey than it would be if you were to mail people to their homes and send them a paper-based survey. You can probably make a better guess at who people are and how you get a representative sample. So that moves people into all sorts of statistical jiggery-pokery where you try and figure out “Well, okay, I got all these people to fill it in, what can I extrapolate from this? I’ve only got three young people to fill it in. Can I readjust these figures’ weight so that the number of young people is more akin to what it is in the population?” And that’s when you start to get into potentially dangerous territory. That’s quite a big issue, this idea of representativeness with online surveys.
The future of online surveys
Liam Geraghty: From the user’s side – and pretty much all of us have experienced this from time to time –, badly made online surveys are a headache.
Zoe Sinnott: One thing that really irritates me is when a business asks you something that I expect them to already know about me. For example, they send a survey to your email asking you for some feedback. And then, one of the questions in the survey is what is your email address? And I just think that it’s obviously not personal, and I’m not sure where the data is going. That’s definitely a pet peeve.
Liam Geraghty: That’s Zoe Sinnott again, the senior product manager at Intercom who helped lead the development of Intercom Surveys.
Zoe Sinnott: Another one is when they ask too many questions, right? I’ll answer four or five, probably not more, and I’ll get bored and drop off. And this definitely may be just me, but I’m not a fan of the way some businesses trick you into believing you can take the survey from an email and you think, “Oh, great, the survey is in the email. I don’t have to go anywhere.” And then you click on it and it pops over a new window. And you’re like, “Oh, now I’m annoyed because I want it to be in my email.” I’m not a fan of that either.
Liam Geraghty: So what’s next for online surveys? How do you begin to change something that has become ubiquitous in our online lives, and not always in a good way?
Zoe Sinnott: Intercom Surveys is a new feature that we’ve just launched that allows our customers’ businesses to ask their customers questions, and they can do that in the context of their product, across their web mobile applications. The surveys can be really targeted – you can use all the power of Intercom in terms of the audience rules to make sure those surveys are targeted at the right people. Then, the thoughts, feedback, and sentiments that you capture are recorded and stored in Intercom in real-time, which means that you can take actions based on the data you collect to drive tailored customer experiences.
“They’re in context, which means that you’re asking customers questions about the thing they’re currently doing in your product. And so, you are more likely to get better response rates”
Liam Geraghty: Zoe says this is really going to change how companies collect and implement customer feedback.
Zoe Sinnott: We think of surveys in two parts. So, the first is the actual surveys themselves. We think they’re better than traditional surveys that you might receive via email in a number of ways. First of all, they’re in context, which means that you’re asking customers questions about the thing they’re currently doing in your product. And so, you are more likely to get better response rates and better quality response rates as a result because they’re already thinking about it.
The second thing is that they can reach users wherever they are, be that your web application, your mobile app, and they can also be sent via a link through an email or a push notification if needed be. As I mentioned already, they can be hyper-targeted so you can reach users at exactly the right moment. And they can be customized and personalized to make it look and feel like the brand and match their tone and style so you can do things like change the color and make sure it fits with the page that the survey is being presented on. You can include details about the customer such as their name or their company in the context of the survey itself. And so, it feels way more personal and doesn’t even feel like you’re being surveyed at all.
Liam Geraghty: The other part of Intercom Surveys is the way the data can be leveraged through Intercom to drive actions and workflows in what Zoe says are really magic ways.
Zoe Sinnott: That involves capabilities like being able to store the responses that you are getting as a user attribute in Intercom, and then use that to send users down different paths and campaigns to give them a more personalized messaging experience. You can also use that data to personalize their support experience. And, of course, do reporting and analysis using the data that you’ve collected directly in Intercom, or even send that data to other tools like Salesforce or Slack or wherever your teams are.
We think it’s infinitely better than traditional surveys that were sent via email, which would typically get low response rates from a biased subset of users. The data is not very reliable and it’s normally captured in a tool that nobody has access to, so you can’t really use the data to do very much. It’s pretty much a zero-sum game for both the business that is asking for the feedback and the customers who are giving it.
Liam Geraghty: Zoe says one of the advantages of Intercom Surveys is that it will be right there at the very moment it’s needed.
“What’s incredibly different is the speed at which we can go from thinking of the survey we want to send to actually sending it and getting the results”
Zoe Sinnott: Yeah, exactly. I mean, you think about a really simple experience. You want to find out what somebody’s experience was like during the checkout process – they bought something from your online shop and you can either send them an email three days later and try and get feedback, or you can send a survey straight away right after they have finished and you can get that feedback at the moment. It’s infinitely more powerful.
Liam Geraghty: What do you think people like Herman Hollerith and Charles Seaton would make of how far we’ve come in this field?
Zoe Sinnott: The surveys themselves haven’t really changed that much. Obviously, they’re more engaging and beautiful, but the nature of a survey is the same. What’s incredibly different and what they’d be really shocked by is the speed at which we can go from thinking of the survey we want to send to actually sending it and getting the results. I imagine that took them months or years to do. And then, obviously, the way that we can connect the data and automatically leverage it to drive really powerful actions. I think that’s something that they’d be amazed by, too.
Liam Geraghty: My thanks to Andrew Whitby, Rick Klau, Tristram Hooley, and Zoe Sinnott. You can find out more about Intercom Surveys and what it can do for your business on intercom.com. The only thing left for me to do is to finish filling out this census form. We’ll be back next week for more Inside Intercom.
