Mine the Hidden Treasures From Open-ended Answers in Surveys

Surveys can be powerful tools as they can unearth challenges, concerns, and suggestions from respondents. When developing a survey, we often solicit an open-ended response at the beginning before the respondent has had the chance to read about specific topics of the survey. Their feedback is therefore more likely to be spontaneous and unbiased. 

---

Figure 1:
Word Cloud with responses to “Describe your overall satisfaction with the organization’s IT area”

But how can you use this information? How can you make sense of it all? What is significant? And what is anecdotal? There are a few ways we quantify and illustrate this information. 

One way is to identify key words and/or key themes mentioned repeatedly throughout the threads of responses. From the frequently repeated words, a Word Cloud can be created, with great visual impact showing just the positive feedback or the negative feedback, or both. As the example above illustrates, “Helpful” was the key word used most to describe the IT Services of an organization, followed by “Responsive” and “Great.” The most common negative key word/key theme among the open-ended responses was “Slow Response” followed by “Takes Too Long” and “Insufficient Coverage.”

---

A second way to quantify open-ended feedback is to categorize responses into all positive responses, responses that have both positive and negative content, and finally responses which are only negative. In the figure below, we were able to trend, over multiple years of open-ended responses, the proportion of positive, both positive and negative, and only negative responses to the statement, “Please give us your feedback about the IT area as a whole, and/or on specific IT services.” By trending the respondents’ perceptions of IT services over time, we had another data point to measure the success of IT efforts and services. If you see significant change in the responses over time, this may be linked to some other transformations within the company such as a major, all-consuming project, a large organizational change, etc.

Figure 2:
Graph trending perceptions of IT over the years (based on open-ended response)

---

​There are other ways to quantify and/or illustrate open-ended feedback. The key is to recognize and then capitalize on the tremendous value of open-ended responses in surveys, which, with a small amount of effort, can be extracted, quantified, and acted-upon.