One Survey Question, Six Ways | The Data Coach

Transcript:

[00:00:00] I need a beauty YouTuber to come to my house and explain lighting to me because every time I try to do a setup or use something slightly differently, I look, can you even see,

I look like a human being now, but this could change literally every second.

Hi everyone, my name is Lindsay.

I am The Data Coach. Today we're gonna talk about how to write one question six different ways. My goal is to show you the different question types available to you when creating a survey in Google Forms, SurveyMonkey, whatever it is that you're using, along with their pros and cons, so you can make informed decisions based on your project goals, objectives, and timelines.

If you are new here, hi, welcome. The mission of the Data Coach is to help nonprofit organizations better organize, analyze, and use their data to achieve their missions. One of the ways that we do that is by providing free education and training here on our [00:01:00]YouTube channel.

So if you find this content helpful, please subscribe to join this growing community.

Okay, let's imagine this scenario. You are the director of anew financial education program, and you just wrapped up your first year of implementation. You wanna send out a survey to see what your participants thought about the program and how you can improve for next year.

You have your questions ready, but when it's time to entereverything into your survey program, your question options are a little bitoverwhelming. Does it really matter if something is a multiple choice versus acheckbox versus an open-ended? Yeah, it can. So let's break down some of thecommon question types and when you might use each of them.

The basic question we're gonna work with is, which of thetopics in our workshop were most helpful for you?

The first question type is our multiple choice, a [00:02:00] classic, it looks like this. Which topicfrom our workshop was most helpful for you? Creating a budget. Understanding credit and debit, setting financial goals or making investments. Now, thebeauty of a multiple choice question is its simplicity. Survey takers provideyou with one answer and one answer only, and that makes your analysis reallystraightforward.

And typically you'll have a pretty clear winner at the close ofyour survey. The downside is that some people may have a clear favorite, butothers might have a top two. Or maybe somebody thought that all the topics wereequally helpful in their own way. By limiting people to just one answer option, you don't get the full picture of their experience. You might miss out on giving yourself even more credit for the great work that you did onthis program.

Second option is our checkbox. Which of the topics were mosthelpful for you?

Choose all that apply, and [00:03:00]you can see all of our options here.

A checkbox question allows people to quickly choose more thanone option. This question type can be great because it allows people to shareall of the things that they found the most helpful, the least helpful, thebiggest priorities, their least favorite things about the gala, you know,whatever your question is.

You can get a fuller picture of people's thoughts andexperiences when you give them more flexibility in answer options. However, giving people more options does make the analysis a little more complicated.

We won't really know if all the topics chosen were equally helpful, or if one was more helpful than the others. So that's something that'simportant to you, a checkbox may not be the right fit. And in this example, werun the risk of what's called a social desirability bias. That happens whenpeople give us the answers that they think we wanna hear.

So. If a participant really likes their [00:04:00] instructor or they really like you, the director or the organization, they might choose all of the topics in this question because they wanna make you or the organization happy, not, because that's how they really felt about all the topics in the workshop.

We can remedy this by limiting the number of choices a bit. So maybe choose two, instead of choose all that apply. But it's something to bemindful of when we're selecting our question types. We don't wanna choose things that might unintentionally sway people to give us different answers thanthey would otherwise.

The third option is a ranking question, so that looks likethis. Please rank the following topics from the most helpful, one to the leasthelpful, four. Now, a ranking question can give us really valuable feedback ifwe're interested in comparisons among topics.

In this case, it allows our survey taker to consider all theoptions, compare them to each other and create this order of [00:05:00] favorability, something we couldn't dowith our checkbox option. Now there is a little more complexity to the analysisand ranking questions aren't appropriate for all topics.

Also, if you use this type of question, you'll want to limitthe number of topics or items you want people to rank. A list of five is mucheasier to order. Then a list of 25. Yes, I have seen surveys with 25 topics to rank. Please don't do it. But when it comes to asking about likes, dislikes, priorities,the ranking question can be a great option to capturing even more nuance anddetails of people's perspectives.

Fourth option Likert Scale. Likert scale questions ask surveytakers to judge individual options or topics across a five point scale, thoughsometimes you'll see more than five. Think of rating a restaurant's food onYelp based on the five star system. In this [00:06:00]case, Likert scale questions might look something like this.

On a scale of one to five, one being the least helpful and fivebeing the most helpful, how would you rate each of these curriculum topics?

This is a nice option, if you care less about ranking a set of subjects against each other, but wanna know what survey takers thought about individual topics or issues or experiences. Now, one of the critiques of a Likert scale question is that one person's three might be another person's four. I generally don't worry too much about that, especially if there are a lot of people responding to the survey. It's something you might wanna note in a final report.

Also, some advice, if you are going to use this question type,please do so sparingly. I've seen entire surveys that are only Likert scalequestions and they are very boring and annoying to take.

When people are bored, they're annoyed. Or pressed for time,they are more likely to rush through [00:07:00]their responses or just skip questions entirely, and that can be bad for ourdata quality. So be judicious if you're choosing this question type.

Fifth option is the matrix question.

Unfortunately, this is not Keanu Reeves reading your survey questions aloud in his Neo voice. A matrix question looks like this. Pleaseshare how helpful you found each of the financial education topics in the table below.

Matrix questions can provide you with a wealth of insight aboutwhat people think about a set of related topics. You'll be able to easily seewhat people thought of each topic and quickly compare it to the others. It's also a nice visual break for survey takers.

Seeing a table after so many multiple choice or checkbox questions can help refocus their attention and help them engage with your survey in a different way.

Downsides? Similar to ranking questions. Don't use this question type if you have a lot of topics you wanna cover, [00:08:00] people will get bored after a while.

Seeing a list of 10, 15, 20 options to judge in this table can be overwhelming visually, causing them to either skip the questions or rushthrough the responses and potentially making mistakes along the way. When we're choosing our question types, we wanna make sure we're doing as much as we canto help people avoid giving us the wrong info or not telling us anything at all.

And then finally the open-ended question where we allow peopleto say whatever they want, however they want, , that may or may not be ontopic. Here's an example. Which topics are most helpful for you and why? Ithought that creating a budget was the most helpful because no one has evertaught me how to do that before, and it was much easier than I thought it wasgoing to be.

Now the reason so many nonprofits love open-ended questions isbecause they can be really rich in detail and allow our survey takers toexplain why they [00:09:00] think and feel theway they do. The why is incredibly important when it comes time to makeinformed decisions about programming changes, design, outreach, communicationstrategies, and structure.

And as someone who loves qualitative data, it's even hard for me to say no to an open-ended question on a survey. But there are a few thingsthat can go wrong here. One, there can be a lot of variations in responselength. Some people will give two word answers that don't help you very much.Other people will write novels. The approach to the analysis. There is an artand science to analyzing qualitative data that is fundamentally different thananalyzing numbers. If there's no one on staff that has this background, it'ssomething that's really difficult to do well.

And then the third issue is time. Open-ended responses take alot more time to sort, to read through, and analyze. If [00:10:00] you and your team have the time to do this, fantastic.But if not, these may not be the best question options for you.

What I would say is use your open-ended questions strategicallyand sparingly if you use them at all.

You can always start your project with a survey and if you needmore information, you need more detail, you can reach out to people for quickfollow-up interviews.

So that's it. One question, six ways. In general, you want to use a variety of question options to keep your survey takers interested andengaged, while also choosing the question types that meet your research needs, analysis skill level, and of course your project timelines.

If you have any questions about survey questions , leave themin the comments below. I would love to hear from you and help you out andthanks so much for watching. We'll see you next time. Bye.