Eight steps to defining what goes into version 2.0 and beyond!

No matter whether you’re on version 1.0 or 10.0, the list of potential enhancements is always greater than the available resources.The challenge software companies and product managers face is to identify where they should invest their limited resources to maximize their return and minimize their risk – especially, since most software companies have a fairly limited understanding of what their customers want.So most companies use one or more of the following procedures to decide what to put in the next release.

  1. Make up stuff
  2. Collect list of known bugs
  3. Informally survey customer support and sales to get a better understanding of what customers are saying
  4. Survey actual customers
  5. Rely on user advisory board

Unfortunately, while all these processes are good starting points, they all have their problems, including:

  • Opinions are like elbows, everyone has one or more of them. Whoever is closest or has more power, can make decisions to their liking that may or may not be what customers really want.
  • Oftentimes it’s difficult to quantify what’s really important to customers. Major bugs in features people rarely use maybe less important than seemingly minor issues in frequently used components.
  • Items that irritate but don’t actually stop system from being used, e.g. software is slow, may never get reported to customer service.
  • Customer service is traditionally focused on solving not documenting the problem.
  • Difficult for visionary customers/ users with great ideas to give suggestions. As a result – may never receive many great ideas.
  • Without clear-cut information, i.e. quantitative data, it’s very difficult to identify where should invest resources.
  • Once software is built, hard to determine whether changes actually satisfied dissatisfied customers.
  • Advisory boards may not actually reflect your typical end users.

As a product manager, I too have struggled with many of the same issues. However, after having the good fortune to work at a company helping Microsoft, Yahoo! and others capture and analyze customer feedback I’ve put together an eight-step program designed to help you identify the most important issues, verify your enhancements and validate your decisions to senior management.

1. Make it easy for your users to give you feedback while they’re using the product!

Why not transform your entire user base into a focus group? IBM recently stated that over 30% of their best ideas came from customers – and since your users are the one you have to please why not make it easy for them to give you feedback from within your product?

Microsoft, Yahoo! Intuit and Real Networks are already beginning to transform the concept into reality with significant results. They’ve all begun putting open-ended feedback forms right into their products – and as a result are receiving tens of thousands of comments each month about what customers love, like, dislike, absolutely hate, suggestions for enhancements, etc.

While in the past, the prospect of analyzing thousands of open-ended customer comments was overwhelming, new customer feedback solutions have transformed oceans of messy data into simple reports with powerful drill down features.

By giving the customers the ability to easily give feedback, Microsoft was able to quickly identify a seemingly minor issue was actually the number one source of customer complaints.

Customers couldn’t resize the reading pane in Microsoft’s new “Web-based” Live Mail product – as a result they complained loud and vociferously. The product team was then able to quickly escalate and fix the problem, which lead to a dramatic drop in the negative feedback.

Having sat through many discussions and arguments about what to fix and what not to – I can only imagine that a seemingly minor usability issue like this would have either been altogether ignored or downplayed in favor of a more quantifiable feature without the ability to capture and quantify open-ended feedback, but by having the hard data to show that it was indeed the number one complaint it eliminated any debate about the importance of validity of the enhancement request.

After working as a consultant at Island Data, which makes customer feedback solutions, and interviewing Microsoft, Yahoo! and others, here are a few recommendations to increase the quantity, quality and immediacy of your feedback.

Put a link to an open-ended feedback form directly in your Web site or application: That way customers can give you feedback when they want to – as they’re actually using the product so you’re able to capture immediate feedback.

Make the feedback link as visible as possible: The easier it is for people to give feedback the higher the probability of receiving feedback.

Encourage feedback: Don’t just rely on people remembering to give you feedback. Ask for feedback. Send out emails encouraging customers to give feedback. Offer rewards for best suggestion, etc.

Ask whether the user would recommend the product or service: Fred Reicheld’s ground breaking research on the use of the net promoter score to predict company earnings and profit should be used to help you predict both renewals and new sales. Visit www.netpromoter.com for more info.

Limit the use of closed-ended questions: If you survey users, keep the number of questions to less than 10 – while you may have specific questions you want answered – your end user may just want to give you a very specific suggestion and doesn’t want to be bothered with answering irrelevant questions – and just make sure you always include an open-ended question and a big open-ended text box.

2. Capture as much additional data as possible: If a person is logged in, you can capture all sorts of additional data about them, including their email address, company, industry vertical, demographics, etc…. Even if they’re not logged in, at the very least you can capture the browser, OS, and other information to use for further analysis.

3. Use new customer feedback tools to simplify and enhance the analysis: Depending upon the size of the customer base, you may require additional technology to simplify the process of capturing and analyzing feedback. There are hundreds of different survey companies – but there are very few companies that have good tools for analyzing open-ended feedback. Two examples are

Island Data: Uses artificial intelligence to categorize open ended feedback. Currently used by Yahoo! and Microsoft.

Informative: Used by Intuit, HP and others, gives customers the ability to rank other customers suggestions.

4. Analyze feedback to identify what customers are saying.

  • Who is saying what? Segment your customers/users appropriately so can weigh the value of the complaint, e.g. complaints from secondary markets may be less important than complaints from core market users.
  • Categorize the suggestions and complaints? Try to group the comments together so you can begin to rank issues based on both the number and strength of the complaint.
  • Combine the structured data with the qualitative feedback to identify trends/root causes: Are the customers complaining about the system being slow all using dial up, on older OSs or have some other commonality that may point to the underlying root cause.

5. Dig deeper: Follow up with customers that complained to understand how important issue is and get more details on the issue. Just knowing that an organization is listening – even if the problem hasn’t been resolved is enough to increase customer satisfaction and may prevent customers from straying (at least temporarily).

So whether you choose to call, send an email survey or invite the user to a focus group – it’s important to both recognize the feedback and ask for more details so you can more quickly identify the root cause, understand the seriousness of the issues, gather potential solution ideas.

6. Test: Once you’ve decided what to fix or add for the next release, share your prototype specifically with customers who complained.

After all, since these are the people who care most – the people who are giving you an F and making most noise – shouldn’t you make sure your proposed solution actually receives their stamp of approval before investing significant resources.

And as most of us can attest to and research shows, customers feel much more positive about companies and people that went the extra mile to fix a potentially bad experience, I can attest to the power of going the extra mile.

Shortly after I bought the laptop – it had problems. Instead of being told to take a number, wait x days, etc. they worked on it while I waited. And then when that fix didn’t work, they gave me a docking station for my trouble – and replaced another piece of hardware. Since then, I haven’t a problem – but if I do, I know they’ll be there for me. And as a result, I always recommend them to anyone looking for a computer.

Once your complainers have approved the enhancements, test the prototypes on other customers to validate whether your complainers feedback matches rest of your customers.

7. Validate enhancement/investment: Once the new version is in the marketplace it’s important to validate your decisions and close the feedback loop. Depending upon the resources and size of your customer base – you can have a series of survey questions for everyone – and then create a subset of questions addressing issues that were specific to certain users.

Creating both general and more specific surveys will help you understand the impact of you changes on both the general population and the people who specifically complained about certina issues and about the product and company as a whole. Not only will this help you validate your investment, it’s giving you another chance to get more feedback and deepen your relationship with those customers.

8. Present analysis to senior management: What better way to establish your credibility than to be able to present the results of your before-and-after surveys. Since traditionally the enhancement decision-making process is pretty opaque to senior management – being able to show how and why the decisions were made and ultimately the results, higher customer satisfaction/net promoter scores will be huge for senior management.

And, if in the past, sales has always been skeptical about the value and viability of new features, being able to show customer feedback explaining the value of the enhancements will help you kick start the sales and marketing process.

Just remember, it all starts with making it easy for customers to give you feedback and actively encouraging feedback.


I help companies successfully develop and market products. I have over 10 years of marketing and product development experience in a wide variety of industries and is currently available on a consulting or full-time basis.

Contact me at (858)337-2727 or kevin@kevinmireles.com to find out how I can help your organization.


  1. Marion Saffery says:

    Hi, Kevin,

    I read with interest your “Eight Steps” article. I have a related question.

    One often reads statistics–albiet widely varying ones–stating something along the lines of one customer complaint represents X number of people who feel the same way.

    Is there such a statistic for positive customer comments. Does one positive comment represent a certain number of people who feel the same way?

    Your insight would be appreciated.

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s