Ranking Scale vs. Rating Scale: Which Is Best For Your Survey?

[…]

How to Use Rating Questions vs. Ranking Questions in a Survey

You’ve spent a lot of time deciding what to ask in your next market research survey, but have you considered how you want to ask a question?

Deciding on a question type, especially between a ranking scale vs. rating scale question, vexes many researchers. After all, the way a question is asked, and its format, influences your results and how you can use your data. 

This post provides a ranking and rating scale definition, examples of each question type, and tips for deciding between the two common question types used when gathering consumer insights.

Recommended: What’s a Good Sample Size in Market Research?

What is a Ranking Scale? 

A ranking scale forces respondents to rank a list of items with only one selection in each rank. Ranking scale questions often ask respondents to rank based on preference, but you can get creative with your ranking criteria. For instance, you can also ask people to rank new features based on importance or how likely they would buy a product. 

Ranking Questions Examples

Example 1:  Rank the following snow cone flavors from 1 to 5, with 1 being “keep them coming” and 5 being the “no way.” 

• Grape
• Cherry
• Lemon
• Strawberry
• Orange

Example 2: Rank the following items by how much they influence your choice to buy a car, from that’s it (1), not so much (2), and don’t care (3). 

• Price
• Gas Mileage
• Cargo Space

 

When to Use a Ranking Scale Question

If you’ve ever had the age-old debate of where to eat with a friend or significant other, you’ve probably been in a scenario where you should have used a ranking scale question. The conversation always starts innocently enough. “Hey, do you want to grab some lunch? How about tacos, sushi, or hamburgers?”

Then comes the dreaded response that starts the inevitable restaurant indecision spiral: “Oh, I don’t care. They all sound great.” 

Your question did not yield any actionable data. Tacos, sushi, and hamburgers were all acceptable choices for you. You didn’t need to know if all three were acceptable to your lunch buddy. You needed to know which one was their top choice. 

Use a ranking scale question when you need to know customers’ likes and dislikes compared to one another. Force-ranking questions require respondents to give some thought to their preferences. Going back to our snow cone example question.

Let’s say you’re trying to narrow down your offerings from five to three flavors. If everyone tells you they like all the flavors, that doesn’t help. But if you require them to rank them and most people rank lemon and cherry at the bottom, you know which flavors to cut from your menu. 

 

Limitations of Ranking Scales

Ranking scale questions do have some limitations. Ranking scales are not as helpful with longer lists of items. If you give people more than 5 responses, you could run into issues. 

First, the longer your list, the more likely respondents are only to have extreme opinions about the best and worst options. You probably can’t gain much insight from what people put in the middle tier. Another obvious limitation of this type of question is that while it will reveal what most people prefer, it will not explain why they have that preference. 

Ranking scales for long lists also can frustrate respondents, especially as more people take mobile surveys on their phones. Using drag-and-drop to reorder or entering a number next to each item is cumbersome on smaller screens. Also, if people like some things equally, they may resent being forced to rank them.

As a best practice, try to use ranking scales sparingly in any one survey. You can also try branching questions where you ask a follow-up question based on their answer. For example, you could use an open-ended text question to ask why someone ranked an item first or last. 

What is a Rating Scale? 

Rating scales are one of the most frequently used question types for market research surveys. These questions ask people to pick a value to describe their level of satisfaction or agreement with something. They also can be used to measure purchase frequency. 

Examples of Rating Scale Questions

You’ll likely recognize two of the most common types of rating scale questions: NPS (net promoter score) and Likert scales. 

NPS Example

On a scale of one to ten, how likely are you to recommend (your brand here) to a friend or colleague?

very unlikely – 1 2 3 4 5 6 7 8 9 10 – very likely

Likert Scale Example

Rate your level of agreement with the following statements:

I decide to buy a car based on the price. 

Strongly agree | agree | neither agree nor disagree | disagree | strongly disagree

 

When to Use a Rating Scale Question

Rating scales help determine people’s attitudes in a way that is easy to ask and digest. Reasons to ask a ranking scale question include:

• Getting feedback on proposed changes to a product
• Measuring people’s attitudes or perceptions of your brand
• Gauging people’s opinions on current events

 

Limitations of Rating Scale Questions

The biggest drawback to ranking scale questions is a phenomenon called satisficing. People don’t put effort into evaluating your questions and give everything the same rank because it’s easier. Remember our lunch example? Saying all three choices were fine is an example of satisficing. Results from rating scale questions are often too similar to help a company choose between multiple options. 

 

When to Use Ranking Scale vs. Rating Scale

Deciding between a ranking scale vs. rating scale question can be difficult, but when you start to understand the limitations of each question type, you’ll see there is a place for both in your research. When you need to determine general attitudes and feedback, use a rating scale. If you need to determine priority, use a ranking scale. 

Here’s how you could use both questions types as a part of your product development process. Let’s assume you’re designing a new smartwatch.

1. Use rating scales to ask about potential features. You don’t care about ranking them yet; you just need to know what potential buyers want in a smart swatch. 
   1. Rate the following smartwatch features from who cares (1) to must have (5). 
      1. Heart rate monitoring
      2. Workout tracking
      3. GPS
      4. Stopwatch or timer
      5. Voice activation
      6. Cellular connection
      7. Text messaging
2. Use the results to narrow your list down to the top contenders for development
3. Use a ranking scale question to ask people to rank their preference for which features would make them more likely to buy or use the product. 
   1. Rank the following smartwatch features from 1 to 4, with 1 being the most important thing you look for in a new watch. 
      1. GPS
      2. Heart Rate Monitoring
      3. Voice Activation
      4. Text Messaging
4. Use the results to determine which feature or features to develop first. 

To see how quickly you can start asking rating and ranking questions on our mobile platform, book a demo with one of our OnePulse research experts.