This article is for: startup founders who are developing a first version of their software product. User feedback and usability testing help ensure the product design, UX/UI and functionality are aligned with customer needs.

Written by: Alex Senson, Ashley Burton, Tyler Boulanger

<– Back to: “How to Create Software: Development Techniques to Learn”

---

No matter the stage of development your startup is at for your software product, you need to ensure your startup is collecting user feedback. How else can your startup be sure that they are creating the best possible product for their users?

In this article, we will outline the importance of collecting user feedback. Additionally, we will cover the best time and method of gathering this feedback, along with how to prioritize your startup’s findings.

This article is for startup founders who are currently undergoing user testing or are looking into different methods of validating the user value of their product. This article will cover the key differences between usability testing methods for early development stage, MVP validation and late development stages, and post-commercialization. This article will help founder learn about user feedback, and the most effective ways to collect and implement it to improve your software product.

You should be thinking about how to collect user feedback on your product right from the minimum viable product stage. Use this resource to learn about importance of usability testing, tools that can be used to collect feedback, along with how to prioritize your findings. Ultimately, the goal is that through learning about usability testing your startup will be able to create products that customers are eager to purchase.  

Topics to be covered

• Introduction
• The Importance of User Feedback
• Collecting User Feedback at Different Product Stages
• Tools That Can be Used to Collect and Analyze User Feedback
• Understanding User Feedback
• Prioritizing User Feedback
• When to Make Changes Based on Feedback
• Overall Tips on How to Get User Feedback Successfully

We will illustrate these ideas through case studies of companies and founders who work with the Altitude Accelerator network

• Case study: Collecting user feedback to increase product adoption at Prodigy
• Case study: Using free tools to gather user feedback at Lucky VR

---

Introduction to user feedback and usability testing

Key Takeaway: Usability testing involves providing users with a version of your software product to test and provide feedback on based on their experience. User feedback is collected at all stages of development (ideation, MVP, later development, and post-commercialization) and needs to be prioritized to ensure continuous product improvement.

---

The only way to truly understand if your startup’s software product is providing value to the user is to collect feedback. This ensures that the product being developed is valuable to the user’s workflow. Startups can collect this feedback by providing users with a version of the software product, such as a wireframe, clickable prototype, interactive prototype, or a live product.

Usability testing is primarily conducted during the Minimal Viable Product (MVP) and later stages of development. In the ideation stage, there are slightly different ways of conducting customer research in the absence of any tangible product. This is often called ‘customer development’ and involves primary and secondary market research. Altitude Accelerator covers this topic in its Startup Bootcamp event. Once the product has been launched commercially, user feedback is continuously conducted until the end of the product’s life.

User feedback informs the design process which can help create product stickiness, affording a competitive advantage. This is especially important for software companies because they are constantly adding new features and improving user experience. User feedback also ensures that every possible point of resistance to using the product is eliminated or reduced to improve the user’s experience.  

The overall goals of collecting user feedback are; product iteration and customer-driven insight into design. This is especially important when startups are using the lean development methodology that includes an iterative development process based on rapid feedback and improvement cycles.

The importance of user feedback and usability testing

Key Takeaway: User feedback should be collected to identify and better understand the pain points of the user. This information can be used to better understand existing analytics data and identify ways to improve your startup’s software product.

---

Usability testing helps startups understand the WHY behind what people are doing. Why are people using one feature so much more than another? Why do most of the users stop creating accounts on the last step? What causes users to user your product less frequently and eventually stop altogether?

Usability testing can be used to help answer the following questions:

• What are the characteristics of your users (for example their demographics, psychographics or professional status)?
• Why they behave a certain way?
• What do they think you should improve within your software product?
• What do they think about your competition?
• What frustrates users the most about your product or your competition’s?
• What are your user’s particular points of resistance to either adopting or using your software product, or your competitor’s software?

Collecting user feedback is beneficial to startups as it provides context to what their analytical data is showing them. Some of the benefits of collecting user feedback include:

• Users feel heard and empowered when changes that they recommended are implemented, improving their experience and increasing their trust in your startup
• Eliminates the guesswork in product development or enhancement because changes to the product are driven by user-insight, not just the ideas of developers
• Save time and money during development because all development work is data-driven based on user feedback

Real innovation comes from constant iteration, and user feedback is essential for this process. By collecting this valuable feedback, startups are able to create a product that users are eager to purchase.

Collecting user feedback at different product stages

Key Takeaway: User feedback can be collected during early development stages, MVP validation and later development stages, and following the commercialization of the software product. Each stage as different user feedback methods that should be used based on the type of product being provided to the user for review.

---

Startups should collect user feedback throughout the entire product lifecycle. This includes collecting feedback during early development stages, MVP validation and later development stages, as well as post-commercialization. Each of these stages of the product’s lifecycle can yield valuable feedback and can help startups create a software product that users want to use.

Early development stages

Key Takeaway: User feedback can collected during early development stages; however, it can be difficult without the creation of an interactive prototype. Startups should utilize lean development methodologies to create iteration cycles focused on testing and user interviews.

---

User feedback can be collected during early development stages (ideation, prototyping, validation) though it can be difficult. Unless you startup has created a solid clickable demo, or an interactive prototype your startup is simply discussing an idea with the user which is not the most productive way to use your limited resources. In this case it’s better to employ customer development methdologies – essentially having conversations and interviews with potential users and customers to deeply understand their pain points and feelings about existing solutions (or lack thereof). The key to this is asking the right questions.

For early stage validation, startups should follow the lean development methodology, including tactics from The Mom Test: How to Talk to Customers & Learn if Your Business is a Good Idea When Everyone is Lying to You by Rob Fitzpatrick and The Lean Startup: How Today’s Entrepreneurs use Continuous Innovation to Create Radically Successful Businesses by Eric Ries to collect relevant user insights.

Insights from these books and methods focus on how to better understand the wants of the target users through interviews and research discovery. These methods can be used to better position your startup for quick testing and iteration cycles rather than focusing on large chunks of development with no chance for conducting usability testing. If a startup has a clickable demo or an interactive demo created at this point they can use the methods and timings recommended within the MVP validation and later development stages section below to gather user feedback on their software product.

Minimum viable product (MVP) validation and beta development stages

Key Takeaway: Startups can collect user feedback during the MVP validation and beta testing stages through the use of in-person and remote feedback methodologies. The goal of this feedback is to allow for quick cycles of iteration and testing ensuring that the developed product is free of any pain points the user may experience, or minimized as much as possible.

---

Once a tangible product has been created, user feedback can be collected based on in-person observation and remote feedback methods to evaluate the MVP. This feedback is collected to test the technical questions a startup has surrounding a how the users interact with the software product, as well as test the viability of the product within its anticipated market. The effectiveness of UX and UI design elements can also be evaluated.

User feedback and usability testing methods with a tangible product

During MVP creation and beta testing startups can collect user feedback in a variety of ways. Methods used during these stages of development are a combination of in-person and remote feedback collection options. The following are some of the different ways that startups can gather feedback at these stages of development:

• In-person observation — Startups can generate insights by observing the user sitting at a computer using the software product. This allows startup team members to observer the user while using the product and make notes of their interactions with the product.
• Feedback boxes — These can be used to receive feedback on minor annoyances and issues that bother users to identify common trends in the way users feel about things. By engaging with the user, startups can identify trends based on large numbers of users in real time and address and systemic problems identified.
• Make development ideas public — Startups can tell users some of the development work planned for later stages of iteration. This allows startups to gain insight into whether the ideas being worked on are the right ones.

When to collect user feedback during MVP validation

During the MVP validation and beta testing stages of a startup’s software product user feedback can be collected using the above mentioned methods. These feedback methods should be used during the following times to ensure that feedback is collected effectively:

• After the user is observed spending more time than expected on a step or screen. This means that there could be confusion around how to complete the task, or that someone unexpectedly drew their attention.
• When they are about to leave the screen ask them about their satisfaction, or if they need help figuring out how to complete the next step of the task.
• After they have completed specific tasks within the software to determine if there were any challenges completing the task, or steps that they found unnecessary or confusing.

In summary, usability testing can be conducted during the MVP validation and beta testing stages of a software product’s life cycle. Startups should utilize both in-person and remote feedback methodologies to ensure that the product being created will be purchased by the anticipated users.

Post-commercialization user feedback and usability testing

Key Takeaway: Usability testing can be conducted during the commercialization phase of a software product through the use of remote user feedback methodologies. Startups should utilize feedback boxes and always-present chat features to receive feedback to ensure that the developed product remains competitive within its ever changing market.

---

Post-commercialization of a software product startups can continue to collect user feedback on their product. At this point the product has already gone through a number of iterations during development based on user feedback previously gathered. User feedback needs to be continuously collected following commercialization to ensure that the product remains relevant in the changing market, and is not outcompeted by its competition.

Post-commercialization user feedback methods

Following the commercialization of their software product, startups can collect user feedback in number of different ways. These methods are conducted with the user not in the same room as members of the startup’s team. The following are the ways the startups can gather feedback after their product is available commercially:

• Onboarding surveys — It is important to gather onboarding feedback so your startup can identify if the user was dissatisfied with any part of the onboarding experience. This ensures you are kept in the loop in order to better understand how the process can be improved for future user.
• Always-present chat — Users like to be able to ask a live person a question in real-time. Having an always-present chat available allows users to raise their questions and issues immediately. This allows your startup to stay on top of any issues that arise, as well as showcase your customer service capabilities.
• Feedback boxes — These can be used to receive feedback on minor annoyances and issues that bother users.  This is useful as users may not always reach out to the support team every time there is an issue.
• Analytical activity information — Using a third party service, user analytics can be tracked to identify why certain outcomes occur. If users check out all the product’s features during the free trial, but do not purchase the free version startups are able to learn that there was probably not enough value in the product and decided to move on.
• Cancellation surveys — These can be used by startups determine the main reasons for why the user is leaving. This is important as it allows you to develop product features that can interest the user, or fix a commonly experienced issue. Some of the questions that startups can ask to better gauge whey they are losing users include:
  • What was your favourite among the features provided?
  • Is there something about the software product that you believe should be changed?
  • How was your experience with us?

When to collect user feedback post-commercialization

When a software product is in the post-commercialization stage of its life cycle user feedback can be collected during certain situations to receive optimum information. The above mentioned user feedback methods should be used during the following times:

• Do not ask for feedback while blocking an essential action (for example, when on the shopping cart page or during newsletter signups). It is always good practice to not interrupt the user in the middle of their task for the gain of the business.
• Allow users to interact with the software product for a few days before asking them for feedback. This ensures that users have had enough time to experience the product and try out all the features before being asked to evaluate it.
• After a new version of the software product is released allow users to provide feedback to ensure that the added feature did not take away any of the product’s value.

In summary, user feedback can and should be collected following the commercialization of a software product. Startups can use a variety of different feedback methodologies to ensure that information is gathered on new and existing users, as well as users who have decided to move on from your software product.

---

Altitude Accelerator Case Study: collecting user feedback to increase product adoption – Prodigy

Prodigy, a math-based game for children, has user feedback methods built right into the product. By adding these feedback methods into the game, Prodigy is able to continuously gather feedback on how their product performs and the satisfaction levels of their users. A knight character called “Sir Vey” provides surveys for the users to fill out and over one million children log in every day to provide responses.

In addition to in-game feedback, Prodigy conducts in-person visits to classrooms, and other stakeholders such as principles and superintendents. “We have found that talking to the stakeholders prevents you from making assumptions about them,” says Rohan Mahimker, Co-Founder and CEO of Prodigy.

As a result of the analysis of the above mentioned collected user feedback, Prodigy was able to increase the product’s integration into the school system. The collected feedback detailed the state of a school’s math program, standardized test scores, the math curriculum being taught at each grade level, as well as the student’s experiences. By collecting and analyzing all of this user feedback, Prodigy was able to determine the exact features required by the users to ensure product adoption.

Prodigy collects both in-game and in-person user feedback to help guide future development within their company. By continuing to change based on user feedback, Prodigy is able to grow and reach more users within future and existing markets.

---

Tools for collecting and analyzing user feedback

Key Takeaway: Intercom, SurveyMonkey and Square Feedback are all usability testing tools that allow startups to collect and analyze user feedback. Startups should select a tool based on the type of feedback being collected, as well as the method being used to collect it.

Collecting user feedback is not a new concept to startups. Whenever a new product is in development, even just at the ideation stage, work needs to be conducted to ensure the product delivers the right value to the customer.

Though the concept is not new, the methods used to collect user feedback are expanding and modernizing. There are now software and tools available to startups in order to streamline the feedback gathering process. Below are some software tools that can be used to gather user feedback for startups regarding their software product.

Intercom

Intercom provides startups with a series of tools that can be added to the interface of your software product. This can use used to track and report different types of user vitality data (for example, time on a feature, how many clicks on a page).

Intercom can then be used to tailor surveys and questionnaires right from the interface your users are already using based on certain triggers that your startup defines. This saves your startup time and money instead of having to create the surveys in-house.

The product can also run a help desk, or other chat bubbles to get different types of information. Intercom is rated one of the best user feedback tools available, and is recommended by top marketing agencies around the world.

SurveyMonkey

SurveyMonkey is an old tried and true product that continues to help startups interact with their users. The platform allows startups to create, syndicate, and collect responses for a list of questions you pre-define.

These questions can be sent to an existing set of users. Alternatively, the survey can be provided to people outside of your startup’s current stakeholders in order to expand your business.

The SurveyMonkey suite allows for instant feedback and the aggregation of data. All of the collected data can then be used to impact future decisions.

Square Feedback

Square Feedback allows startups to create an automated feedback email to be sent to their users after they make a purchase. This email can be tailored to ask the core questions that your startup would like to have answered. The main purpose of this product is to engage in those people who might have positive reviews or testimonials to give your startup’s software product. 

These types of auto communication can be used to create follow-up emails about a number of different topics in order to gain feedback. Startup can create follow-ups regarding products that were not purchased, or on an anniversary of purchasing the product. This can all be done to answer different questions that the startup may have about the user’s experience.

Overall, there are many different tools available to startups in order to gather and analyze user feedback on their software product. Startups can use these tools to allow themselves to focus more on addressing the feedback received, versus trying to gather the feedback themselves.

---

RIC Case Study: using free tools to gather user feedback – Lucky VR

Lucky VR, a virtual reality (VR) company that specializes in casino games, prioritizes usability testing to ensure that their product remains valuable to its users. The company gathers user feedback through a variety of different methods.

Real-time feedback is gathered through a Discord group, an online chat community for gamers, with more than six thousand users in it. Likewise, there are Google™ forms available to track questions and comments to identify features that need to be revised or added. “By gathering real-time user feedback, Lucky VR is able to quickly identify and address any issues within the product to ensure that our users remain satisfied,” says Jeff Lande, Founder of Lucky VR.

Through a combination of user feedback from the above mentioned sources, as well as automatic error reporting through Unity reporting, Lucky VR has been able to further improve their user’s experience with their product. By ensuring that there are multiple avenues for the collection of user feedback in place, Lucky VR has shown that startups can remain on top of any issues and requested changes without having to invest significant amounts of money. Startups can still collect sufficient amounts of user feedback, all while using free tools and forms, prior to moving into a more comprehensive and expensive way to track and review user feedback.

---

Understanding the value of different user’s feedback

Key Takeaway: New and existing users are able to provide different types of valuable information to startups. New users help startups understand adoptability issues with their product, while existing users help identify features and changes required to keep them.

After startups collect user feedback, they need to analyze it and draw conclusions about their findings. Feedback can lead to changes in your product, communication strategy, or even increased training for your support team so they are more helpful.

Feedback provided from new users can be used to gain information regarding:

• Is the developed software product easily adoptable? This user provides information regarding navigation, screen or application layout, and the main workflow of the product.
• Does the software product meet the expectations of the user? This user allows startups to identify if their product is meeting the minimum requirements of the user for them to integrate the product into their workflow.

The feedback provided by existing users can provide valuable insights to the startup:

• Users that have identified an issue or require a feature and are self-reporting indicate to the startup that there is something significantly preventing a smooth workflow. This feedback provides your startup access to the direct needs of the user.
• Reviewing a user’s time using the software product and what features they use allow startups to gain insights into the user’s current workflow. identifying how users regularly interact with a product helps startups understand the needs of an everyday user.

In conclusion, by understanding and implementing usability testing during the development process startups are able to improve their product and company. Startups need to recognize the different type of feedback that they receive from different categories of users to better understand its value.

Prioritizing user feedback during usability testing

Key Takeaway: Startups receive an overwhelming amount of user feedback during development and post commercialization. Startups do not have the resources to jump on each point of feedback; some suggestions may not even be beneficial to the product, as a result they need to be strategic about what feedback to take action on.

After users have access to your software product they will begin to request new features and changes. As your startup beings to receive these requests you need to create a plan on how to address them. Especially since change requests from different users will start to contradict each other or become too difficult to implement.

As a result, startups need to prioritize usability testing to organize both the startup’s plan for future development, and the user’s requests for new features and changes. Based on user feedback a prioritized backlog, a prioritized list of work for the development team that is derived from the roadmap and its requirements, should be created by your startup.

Startups should review this prioritized backlog before each iteration planning meeting to ensure that the prioritization is correct. This review can also be done to ensure that the feedback from the last iteration is incorporated into the priority backlog.

How to prioritize user feedback

Key Takeaway: User feedback can be prioritized based on the startup’s business plan, a SWOT analysis, the potential gains the change can have, and the number of users requesting the change. Startups should always focus on what the majority of the users are saying to ensure that any changes are beneficial to the majority, and do not create new pain points for the users.

---

Startups can get overwhelmed when trying to prioritize a list of changes and features that the users have requested, along with the planned changes within the development plan. Startups should review a number of different criteria when trying to determine how to prioritize the collected user feedback:

• What changes have the greatest need while taking the least amount of development work? By getting enhancements into the user’s hands quickly the relationship with the user is improved.
• What is in the business plan? Changes and features that can push the business ahead and allow for infrastructure improvements should be prioritized over low impact changes.
• Startups can conduct a SWOT (strengths, weaknesses, opportunities, and threats) analysis of their market. This can be done to identify the changes and features needed to lead the market or fill gaps.
• Review the potential gains from adding certain features and changes and evaluate how much effort it would require to create. This is done to determine the feature’s investment over return ratio.
• How many users are requesting the change? All users will complain that the issue that they are experiencing is the most significant, but if a majority of users share the same opinion it should jump high in the priority list as it will likely address a systemic issue identified by the trend in user feedback.
• Have the users vote on which features or changes are most important to them. This allows the users to feel more in control of the development process, and gives the startup a real idea of what the user values the most.

Overall, when trying to prioritize user feedback make sure to focus on what the majority of the users’ requests. This ensures that your startup is building the most important capabilities for their users first. 

Different ways you can rank your priority backlog

Key Takeaway: Startups can rank a prioritized list of changes and features on either a single scale ranking system, or a severity of annoyance scale to communicate to the development team the priority of the development work. Make sure to select one early on and follow through with it to ensure consistency during the project.

---

There are a variety of ways that your startup can organize its priority backlog. Each tells the development team the priority of the feature or change based on the overarching business roadmap.

One method of ranking your startup’s priority backlog is based on a single scale ranking of one to three. This ranking system prioritizes the features and changes requested based on the following explanation:

• 1 = minor, causes user hesitation and irritation
• 2 = moderate, causes user delays, more irritation, and make sure users fail their tasks
• 3 = critical, causes users extreme irritation and makes all users fail their tasks

Another method of ranking your startup’s priority back log is based on the severity of annoyance that the lack of feature or issue has on the user. This ranking system prioritizes the features and changes requested based on the following explanation:

• Irritant = intermittent or cosmetic issues for the user
• Moderate = somewhat of a headache for the user to work around
• Severe = severely limits the user’s ability to use the product
• Unusable = users will not want to use a particular part of the product 

Overall, there are many different ways that your startup can prioritize the list of features and changes that you have gained from the collected user feedback. Create and stick with a prioritizing system that works best for your startup in order to ensure that you are developing based on an accurately prioritized development plan.  

When to make changes based on user feedback

Key Takeaway: After evaluating user feedback, startups need to make changes to their software product in order to improve it. Ensure that your users know when changes are happening so they feel listened to and a part of the development process.

---

Feedback can be instrumental in guiding future development efforts and uncovering important flaws that even the most development company may not have considered. Startups commonly have either too much or too little user feedback. As a result, startups need to understand how to evaluate when and if they should implement the requested changes.

Some suggestions your startup may receive will come out of nowhere. This is because some users will try to do obscure tasks and demand a solution. These adjustments, however, might actually get in the way of the rest of your users and their workflows.

When reviewing user feedback, all startups need to look at trends within the filtered data. Startups need to identify if there are issues or requested features that are common among multiple users. If multiple users are identifying the same challenges, startups should prioritize that development work first, if it is an appropriate solution, to ensure the majority of the user base remains satisfied with the product.  

Iterative user feedback

Startups need to show their users tangible results and not just record the feedback. A feedback loop with a startup’s users needs to be created to promote a relationship between the user and the startup.

The creation of this feedback loop has two main benefits:

• There is a higher chance of getting immediate feedback from those users, either positive or negative; increasing the chance that you will receive feedback on features and issues you would have never noticed.
• A stronger relationship with the users is built, turning them into advocates and promoters of the product. Having the users’ feedback incorporated into the product encourages them to participate and send more feedback in the future.

Within the feedback loop, startups need to inform the users of changes within the software product. This is because users need to know in advance that the product is changing and how it will impact their workflow. Additionally, this can be done to easily highlight that changes are being implemented as a result of user feedback, promoting others to provide their feedback.

By creating and maintaining this feedback loop, users will continue to provide your startup with valuable feedback. When the feedback is used, users will feel a part of the development process and more likely to promote the product.

Overall tips on how to get user feedback successfully

Key Takeaway: Make sure to ask clear questions to the right users at the right time to ensure that you are collecting relevant user feedback. Remember to always think about the value of the feature or change being requested, if there is not enough value do not commit resources to the change.

Now that you have learned all about user feedback and usability testing, below are some tips in order to collect and use user feedback successfully:

• Ask the right questions at the right time. Ask for feedback on your software after the user has had a chance to use it for a few weeks. This is because users will be more familiar with it, and they will be able to give meaningful insight
• Utilize your channels. Make sure to understand where your users exist so you can open communication channels with them to receive feedback. Determine if it would be best to request feedback from users using social media, support tools, emails or contact forms, product reviews, or user interviews.
• Remember to think about the following question – “what is the fundamental business value behind the request, feature, or task?” Do not commit to building anything until you understand the value behind it and the extent to which your customer base requires that value.
• It is important to understand the users and which ones are providing feedback. Ensure that your startup is targeting the ones that will give you feedback that is most relevant to what you are looking for.
• When performing life cycle management of the product, startups should split their resources between modernizing the product (scalability and stability), and building new features that your users are asking for. It is a delicate balance and ensures a stable and evolving product.
• Feedback should not be mandatory as you may get false answers which can corrupt the pool of feedback data.
• Lastly, remember to maintain a prioritization and evaluation process for feedback. The changes requested by the users and the industry are constantly changing, as a result make sure to change your prioritized backlog to adapt.

Make sure that your startup asks the correct questions at the right time to get the most relevant user feedback possible. By following the above tips and the learning from the rest of this article your startup should be able to successfully collect and understand user feedback to improve their software product.

Conclusion: User feedback and usability testing

The collection of user feedback is important for startups both during and after the development process. User feedback can be collected many different ways and different times in order to gain insight into different aspects of the developed software product.

Though it may cost more money upfront, collecting user feedback will save your startup money down the line. This is because your startup will be utilizing iterative development allowing for multiple cycles of testing and iteration rather that conducting a large chunk of development with limited chance for user feedback.  

Ultimately, user feedback allows startups to ensure that the software product developed is a product the user would purchase. The more feedback that startups receive, the more potential value that the company can add to their software product.

Lessons learned

Congratulations! You have now learned about user feedback and how startups can gather and prioritize them. After reading this article you should have learned the following key insights:

• User feedback needs to be conducted during all stages of development, even after product launch. This feedback can ensure that your software product remains relevant in the changing industry landscape.
• User feedback can be gathered from users using many different methods: surveys, telephone calls, emails, in-product reviews, and each provide your startup with different levels of information.
• Surveys should be the first method of gathering user feedback to identify key trends prior to conducting more in-depth feedback gathering methods.
• Startups may receive conflicting feedback or way too much. As a result, all user feedback needs to be prioritized against each other and the startup’s business plan prior to initiation.
• Make sure to review the value added and the number of users that will benefit from a change or new feature prior to conducting development work.

<– Back to: “How to Create Software: Development Techniques to Learn”