From poorly chosen features for a product to tone-deaf advertisements, sub-par pricing choices, and missteps in choosing an advertising channel, so many disasters can be avoided with proper Concept Testing.
In this guidebook, we are going to dive into the heart of what concept testing is, why you need concept testing, concept testing methods, and examples, basically everything you need to know to run a concept test successfully!
What Is Concept Testing?
The definition of Concept Testing is the process of using surveys to evaluate consumer acceptance of a new product idea before it is introduced to the market. In simpler terms, it’s your key to making sure that your amazing new idea will sit well with your potential customers BEFORE you go through all the costs of creating, marketing, and selling it.
You probably have an awesome idea that you want to make a reality. In your mind, it’s so impressive that you can’t imagine it being a flop. It’s sure to succeed in the marketplace, right? Fast forward to after you’ve spent a heap of money making your crazy new idea a product ready to sell, and you’ve discovered, too late, that you’ve made some miscalculations. Oops.
Concept Testing helps you to gather insight from your target audience before you launch to get a feel for what it will be like when it hits the market.
Why You Need Concept Testing In Your New Product Development
Maybe you need to make adjustments to certain features in your existing product? Maybe you misjudged how much people are willing to pay for it? Maybe the product design isn’t as attractive as you thought? With concept development and testing, you will protect yourself from any nasty surprises down the line.
Companies use concept testing to evaluate:
- A Product Idea
- Product Development
- Different Concepts (A/B testing)
- An Ad Campaign
- Messaging
- The Target Market
- Purchase Intent
- Logo
- Pricing
- Branding
- UX
- You Name It!
Why do you need concept testing in your product development?
It Stops You From Making Bad Choices
Anyone can make mistakes, but how awesome would it be to do a little test run before your slip-ups become something explosively bad?
It Saves You Both Time and Money
The process of concept development and testing makes sure that you don’t end up investing a bunch of money in the wrong things, saving you valuable resources.
It Gives You a Chance to Fix Problems Before You Launch
With concept testing, you can collect feedback on every detail, making sure you know what to focus on and what improvements you can make for more guaranteed success. If you run a successful concept test, you’ll be armed with the raw insight that will help you pick up on your flaws.
This will give you the chance to perfect your concept and make it market-ready!
It Helps You Figure Out What Your Customers Value Most
You are not your customer, obviously, and only THEY can tell you what matters to them. With concept testing, you can better understand what aspect of your product is actually gathering the most attention.
Maybe you thought that one feature was going to be a big hit, but it turned out that a totally different feature, one you thought was boring or insignificant, is really the star of the show.
It Gives You Concrete Proof
An added benefit is that you’ll also be gathering proof of concept for investors, partners, and other stakeholders.
You’ll be able to show statistics, numbers that prove how interested the public is in your product. Without concept testing, it’s just a matter of your own personal opinion.
It Helps You Prepare for Better Marketing
The data you collect from a concept test will help you to estimate your sales, your ROI, your price point, and valuable information about your target audience. You’ll be better prepared for setting budgets, planning marketing campaigns, and knowing which channels to use to reach your target audience.
When to Run a Concept Test
Have you discovered a problem or need? Did you come up with an idea that would be a great solution? Before you move forward, concept validation is your next step.
Concept validation testing is crucial to make sure that you have all the information you need from your potential buyers. This includes their specific pain points (and how well you thought you knew the problem), their priorities, tastes, where they spend most of their time, and so much more.
Are you sure that your solution is as cracked up as you thought it would be? Only your customers can tell.
Don’t limit yourself to the product discovery phase though. You can have concept testing for marketing and prototyping (as mentioned above). The concept testing process is continuous all the way to your final product launch (and even after).
With trends rapidly changing, the best thing you can do is gather valuable customer insight. Bottom line? You should never stop testing, no matter what stage your product is in!
Concept Testing Methods
There are a number of tried and true concept testing methods that you can implement. Mainly, concept tests involve conducting online surveys. This is the best way to get quantitative data on a large scale.
There are four main methods for concept testing. Each one has its advantages and disadvantages for you to take into consideration. Let’s dig a bit deeper.
Comparison Testing
As the name implies, with Comparison Testing, you’ll be offering your survey-takers a choice between two (or more) different concepts in order to understand which concept they like more.
It’s up to you to decide how you’d like to format it. The survey can include multiple-choice and ranking questions, the important thing is that you get a clear idea of which concept is the winning idea. With comparison testing, however, you’ll be opening yourself up for some unfortunate order bias.
Order bias commonly happens in one of two ways. Thanks to how our subconscious minds work, respondents tend to pick either the first option presented to them or the last one. By only measuring how one idea compares to another, you’ll be missing out on some context, like why one idea beat the other.
Since the “whys” are so important for a successful launch, a lot of comparison tests are followed up with monadic testing to get all the juicy details that were otherwise missing.
Monadic Testing
If you want to get the deets, then the monadic testing method might be the way to go because here we aren’t looking at choices but at one single concept.
With this method, you’ll be splitting your audiences into groups. Each group will be shown one concept and then set to the task of analyzing it with a short but extremely targeted survey. You can get really in-depth here without making the survey too long, giving you the kind of context you need about the concept in question.
You’ll be able to ask questions about what they feel about the design and appearance, how much money they would be willing to spend, how they would rate a specific feature etc.
The catch is that, since we’ve split the target audience into groups, you’ll need to collect a large number of surveyors to collect dependable data. A larger sample size (number of people taking the survey) translates into higher costs.
Sequential Monadic Testing
With Sequential Monadic Testing, once again, you’ll be splitting your target audience into multiple groups, but the difference here is that instead of showing only one specific concept per group, you’ll be showing them all of the concepts (in a random order to avoid order bias, of course).
Sequential Monadic testing is great for getting in-depth insights by testing multiple concepts in one go.
Plus, since you don’t need to get an extremely large survey sample size to get reliable data, this method is by far the most cost-effective and efficient way to get the info you need. It’s great for when you are on a budget or when you’re working in a niche market.
Sounds pretty perfect right? Not exactly.
Sequential Monadic Testing has its pitfalls too. Since everyone has to give their insights to ALL of the concepts, this is one hefty survey. And because the survey is so long, the number of people willing to participate can drop significantly.
This could introduce your research to a new kind of bias called nonresponse bias, where your target audience might refuse to participate, or some may opt out of the survey before completing it, leaving you with skewed or nonviable results.
You can try solving this by cutting down on the number of questions you ask, but that will come at the expense of the truly in-depth insights you might need.
Protomonadic Testing
Protomonadic Testing is a bit of a hybrid method. It starts off by first asking questions about multiple concepts and then follows up with the option to choose the concept they like best. This way you can compare the insights collected about each concept with the choice most people voted for.
Do the results add up in the end?
For the most part, it’s just an added layer of confirmation that people really do like the features, characteristics, and details that they say they do. It’s also a great way to reach a solid conclusion about your concept’s strengths and weaknesses.
How to Create Concept Testing Surveys
So, where do you begin with your concept testing survey? What kinds of concept testing questions should you include? Which method makes the most sense for you? Let’s talk about how to design your survey.
Set a Goal
Before taking a single step forward you need to have a very VERY clear idea of what you are conducting this survey for. What kind of answers are you looking for? What kind of data do you need, and how are you planning to use the data that you collect?
Once you decide on a goal, it will act as a reference point for the rest of the stages in product development, including concept testing, keeping you focused. Otherwise, you could be overwhelmed by all the possibilities and lose sight of what matters most.
So ask yourself: What kind of data will be the most helpful when it comes to making a decision about something? How will you analyze the data once you have it? What kind of responses are you looking for?
Brainstorm with Stakeholders
Your next step will be to brainstorm all the different concepts that you would like to test. Involve all the relevant stakeholders in this process, including potential customers, the marketing team, production teams, etc.
Get a list of all the initial thoughts and you’ll be surprised at the ideas that will crop up as the result of so many different opinions brainstorming together.
Set Some Rules
Now’s a good time to also discuss aspects of the survey like how many concepts you want to be tested, what the budget is to get it done, what methodology you want to use, and your target sample size.
You can start asking questions like how much time you want to dedicate to each iteration of concept testing, what kind of design you want the surveys to have, and how you want to implement it.
Think about accountability too. Who is responsible for what aspect of concept testing? This is the time to hash out all of the logistical details.
Create A Consistent Design
Don’t fall into the trap of not paying attention to design. A well-thought-out survey design can greatly affect the answers you collect from respondents. You need to be consistent and extremely clear in terms of both visuals and texts so that your product concept is completely understood.
At this point, the concepts that you want to test should be very clearly defined.
Build The Survey
The hardest part of building the actual survey is making sure that everything is consistent in such a way that will prevent any kind of potential bias. Start off with some context for your respondents so they get an idea of what they can expect. Then, go ahead and describe your concept, and take it from there.
Now is your chance to figure out what concept testing questions you will be asking in the survey. This varies, of course, from survey to survey, depending on what your goals are. But think about finding the answers to questions like how likely will your target audience buy your product? How does your product stand out from the products of your competitors? Which concept feature is a winner and which is a flop?
Ask your most pressing questions about the concept including convenience, usability, functionality, performance, quality, price, values, experience, and so on.
Let’s look at different components to keep in mind when building your survey.
Likert Scales
In some cases, you will want to understand how your respondents rank concepts or ideas. Use Likert scales when applicable to help them rank options. This is a great way for you to collect quantifiable data that can later be used in the analysis stage.
Include Images
It’s best to use high-resolution visuals so that your respondents get a distinct understanding of the concept. Feel free to use images, videos, or even both if applicable. This is especially important when conducting logo testing, packaging testing, or product design testing.
Refer to these visuals in your concept testing questions to avoid any kind of confusion. Your best bet is to remind respondents which concept the question is aimed at with visuals.
Demographic Questions
Make sure you include demographic questions in your survey so that you know that the respondents are indeed part of your target audience. Imagine collecting feedback about your concept that is completely irrelevant because the respondents weren’t part of your relevant audience in the first place! It can totally skew the results.
Mistakes to Avoid
Here are some common mistakes that you should try avoiding if possible.
- Cramming too many ideas into one concept test survey
- Assuming the worst when the responses don’t pan out the way you thought they would
- Assuming that the target audience thinks the same way you do
- Forgetting to tailor the survey to fit different locations
Field The Survey
“Fielding a survey” is just a fancy way of describing the process of collecting your responses. This can be done in a number of different ways including over the phone, by mail, online, on a mobile app, via email, you name it.
Since we are usually working with visuals, phone calls aren’t the ideal way to field a survey for concept testing, but a lot really does depend on factors like your budget, the target audience, the goal sample size, etc.
If you are just looking to get an initial reaction to your super cool idea, maybe do a light test by fielding your survey to the people already in your network. It’s a great way to collect feedback basically for free.
If you are concept testing new features to a product that you already have, the best way to field a survey is to ask your already loyal customers first. The same goes for product price testing, rebranding change to product design, and so on. The opinions of your loyal supporters are by far the most valuable ones that you can ever get.
But let’s assume it’s a brand new concept and you want to really dig deep into your potential market. At this point, you’ll need to get a sample size and target that is representative of your target audience. Know your target well and calculate what sample size would give you the most relevant data. The larger your target audience, the larger the sample size will be.
On the flip side, if the target audience is very niche, you can get away with gathering relevant data from a much smaller sample size.
Analyze the Results
You’ve done it! You’ve gathered all the responses in one place, congratulations! Now it’s time to take that data and turn it into valuable insight.
First, you’ll need to rank the overall performance of each concept. Was there a clear winner? Was there a clear loser? Then try tinkering with the demographics. Is there a segment of respondents whose opinions matter to you more than the others? According to their responses, was the winning concept different?
Compare and contrast data through different filters and pay specific attention to the variables that are more important for you.
Did you get the results you were looking for or did the concept testing process open up Pandora’s box of new questions? Maybe there was a flaw in the concept testing survey that you didn’t realize beforehand. Maybe you need even deeper insight about this or that concept? Or maybe you need to try a different method?
In that case, it’s back to the drawing board, time to buckle up and get ready to prepare another concept testing survey.
Concept Testing Examples
Let’s look at some concept testing examples to get a better feel of how it could work for you. We’ll also include some examples of failures that could have been avoided if they went through the time of properly carrying out concept testing.
Logo Testing
Your logo is how people will remember your brand. It is what will be associated with your company for years to come. It seems crazy not to get a feel for how consumers will respond to it right?
Well, Airbnb certainly didn’t seem to think so, when in 2014 their creative approach to the letter “A” had turned into a Twitterstorm of users comparing it to the already existing logo for Automation Anywhere. Moreover, people had a blast making fun of the logo, saying it resembled, well, genitals.
This whole embarrassing business could have been avoided with logo concept testing, of course.
Website Design
If you are in the process of launching a new website or redesigning one that you already have, then you might want to consider concept testing your website. It makes sense since the website is one of the main points from which you’ll be interacting with your audience.
Advertisements
As a marketer, your biggest nightmare would be to put out an advertisement to the public only to realize that it’s extremely tone deaf. It’s a surefire way to get yourself canceled, and recovery will be pure torture.
Dove is no stranger to releasing controversial ads, only to have to deal with harsh backlash from the public afterward. Sometimes it takes a bit of common sense, but in other cases, advertisement concept testing could have prevented it all.
That was the case for one particular Dove campaign, in which they created six different shower gels designed in varying shapes to emphasize that each woman has different curves.
It turned out, however, that the consumers weren’t too keen on having to check out, “the short fat one” for example, at the register. Of course, Dove would have known that if they simply asked their target market for their opinions.
Needless to say, the limited-edition packaging never made it to the shelves thanks to the backfire.
Pricing
Judging how much to sell your product or concept for can be pretty daunting. How much are your consumers prepared to pay for your product or service? How much extra will they pay for a new feature?
The concept of Google Glass didn’t make the cut for many different reasons, but considering the fact that the concept was so new, and so many people weren’t able to grasp the value yet, the pricing of $1,500 per pair certainly felt way too steep. Something a small concept test could have easily discovered.
Name Testing
Like your logo, your name (and the name of your product) is all part of your brand. It leaves an impression and can be very hit or miss. The name of your company or product can sway consumers, leave a deep-seated impression, and has a big impact on its success in the target market.
So many companies have gone through name changes as they evolved and grew. Tinder was initially named Matchbox, Google was originally named BackRub. Concept testing your name is a great way to find the perfect name that will resonate with your customers.
Messaging
Similar to branding or advertisement, your messaging is key to how you communicate the value of your company, product, and/or services. How well does it relate to your audience? Do they believe you or does the messaging fall flat? Is it clearly understood or is it far too confusing?
Like advertising concept testing, messaging concept testing could be a powerful tool for making sure your message is not only clear but that it resonates well with your target audience.
Another example of tone-deaf messaging that sparked a lot of outrage, this time involving McDonald’s: many people weren’t happy about McDonald’s using the pain of a child’s grief and mourning as an opportunity to advertise their Fillet of Fish. After receiving an overwhelming number of complaints the ad was pulled, and no wonder. Testing advertising concepts with consumers will give you a taste of how viewers will react.
Final Thoughts
Creating a new product is an exciting journey, but with all that enthusiasm and passion it can become very easy to overlook things that could lead to complete failure. With a bit of foresight, planning, and an investment in a well-thought-out concept testing process, you can get rid of a lot of guesswork and increase your chances for a stellar launch!
Concept test surveys are the best way to gather important and valuable insight that will put you on a path to success.
