Using feedback and insights to guide product development

Why use insights?

Collecting relevant insights, and using them to substantiate your ideas, gives your suggestions credibility. It shows that you care about using resources efficiently, and directing your teams’ hard work to where it will have the most impact.

Insights connect ideas back to product outcomes, centering conversations around customer needs and market demands. That helps you gain the trust of your team, customers and leadership, so you can claim the autonomy you need to build a great product. 

Whenever prioritization discussions are derailed by loud or convincing voices, defer to insights to get back on track.

Insights in Jira Product Discovery

In Jira Product Discovery, Insights are an integral part of every idea. Make a habit of adding relevant Insights to ideas with the Jira Product Discovery Chrome extension, our Slack or Teams app, or one of the integrations built by our partners. 

Use Jira Product Discovery together with your support, sales, research and feedback management solutions. We designed Jira Product Discovery so you can add relevant feedback and insights to Ideas, explaining why they’re important, how they should be prioritized, and what they entail.

Then, these insights will be at your fingertips when you start prioritization discussions.

Setting up channels for gathering insights

If you don’t have enough insights to guide your decision-making, it’s never too late to start gathering them. When you have ongoing processes to gather this data, you’ll be able to refine it into the Insights you need to support every product decision.

In this chapter, we’ll explain how to set up direct communication channels with customers and customer-facing teams, and start capturing data you come across during your own research. 

We recommend setting up multiple channels to gather insights:

How the Jira Product Discovery team gathers Insights

The entire process of building Jira Product Discovery was shaped and guided by insight collection. In the rest of this section, we’ll show you exactly how the Jira Product Discovery team set up feedback channels, and used them to inform our product work.

Gathering insights from inbound feedback

Inbound feedback — concerns, comments, and questions for people who are actively using your product — is one of your most important sources of insights.

Collecting feedback with Jira Service Management

On the Jira Product Discovery team, we use Jira Service Management to let customers and end-users share unstructured feedback directly:

Collecting feedback with dedicated Slack channels

For internal stakeholders, we created dedicated channels in Slack and Teams where they can ask questions and make suggestions. We sort through these, then add them to Ideas as Insights using the Jira Slack app.

Collecting feedback with a dedicated Jira Product Discovery project

While the Jira Product Discovery team doesn’t use this method, many of our customers have set up a dedicated Jira Product Discovery project for feedback intake. 

This is a dedicated space, separate from your product and delivery backlogs, where contributors can create their own ideas, add comments and insights to other ideas, and vote on each others’ contributions.

This can be very effective — as long as you have a good way to enforce good practices in the dedicated project. Otherwise, it can quickly turn into a laundry list of small and big asks, with duplicates and redundancies. 

But with careful setup, when the number of stakeholders is limited and the number of ideas manageable, we’ve seen this work pretty well.

Gathering insights through user research

Inbound feedback provides a good view into general areas where your product could improve. But on its own, it’s not enough to help guide your product decisions. In particular, it lacks context. 

From a single piece of user feedback or feature request, you’ll struggle to piece together the whole story: 

• The underlying problem the user is facing
• How important this problem is to their workflow
• How it impacts the rest of the product 
• Whether a different solution could solve their problem

For all of this and more, there’s really nothing better than regular conversations with users. 

Recruiting the right users for research can be difficult. But if you have an inbound feedback or a survey solution, you can use it as a channel to identify users who could be productive research subjects, and get in touch with them to offer a chat.

User research with Dovetail

On the Jira Product Discovery team we mostly rely on Zoom meetings, because we’re all remote and have a global customer base. When a customer leaves an interesting piece of feedback in one of our channels (in-app feedback widget or community group), we contact them with a Calendly link, inviting them to book time with us. 

With the permission of the interviewee, we record these meetings and upload them to Dovetail. There, we can tag important moments in the conversation and add them as Insights to relevant Ideas.

Over time, that’s how we found the lighthouse users, who helped us shape what Jira Product Discovery is today.

User research with research reports

For some topics, it’s better to rely on deeper user research. At Atlassian, we’re lucky to have a Research & Insights team who dives deep in specific topics, using rigorous research techniques to distill them into Insights. 

For example, recently a researcher helped us understand the points of friction for Jira Product Discovery evaluators. We’ve tagged Insights from this report to Ideas in Jira Product Discovery, and are using the outcomes to rethink how evaluators onboard in-app.

User surveys with Pendo

On the Jira Product Discovery team, we use Pendo to create surveys for specific user cohorts based on segmentation and product usage data. 

We have recurring, regular surveys, like a monthly CSAT questionnaire, but also one-time surveys we send to answer specific questions. Typically, these are questions that could have far-reaching product consequences if we get it wrong, so we need clear and timely data.

We summarize our survey findings in a Confluence page, and add them as Insights to relevant Ideas.

Building a community group to gather insights

Community groups can be a very powerful way to assemble users and source feedback. You can tap these groups to answer questions about the product, recruit early adopters for new features, make announcements, share best practices, and more.

Atlassian’s community group

Since the inception of Jira Product Discovery, we’ve relied on the Jira Product Discovery group, a sub-group of the Atlassian community, to engage directly with users. Over time, we’ve seen power users start supporting other users and share their learnings about the product — and we’ve learned a lot from these conversations.

Gathering insights from product analytics

Finally, it’s important to consider quantitative data, as well as qualitative user feedback, when evaluating insights and deciding how to act on them. 

Product analytics can provide insights at two levels of granularity:

1. Growth metrics, like churn rate and customer LTV.
   
   For example, if you notice that evaluators fail to convert to active users and drop off after their first session, it’s very likely you should take a break from implementing new features and take a look at your onboarding flow. 
2. Feature usage and user behavior data. 
   
   For example, if you receive a lot of negative feedback about a feature, and data reveals it’s not used by many people, it’s a good time to discuss whether it’s worth keeping or should be retired.

Using your product analytics tools to glean insights

Obviously, the product analytics tools used will vary from company to company. On the Jira Product Discovery team, we use both Amplitude and Pendo for this purpose. In particular, we use it to track utilization, the popularity of features, and how users interact with our onboarding flows. And again, we add key Insights to Ideas in our product backlog.

Make discussing insights a habit

Product teams make decisions every day. To ground as many of those decisions as possible in customer data, they must work with insights on an ongoing basis. 

Regardless of which feedback and research channels you decide to use, for them to work your team must make collecting and acting on insights a continuous practice. If teams only engage with customer feedback once a month, they’re making a month’s worth of decisions without timely customer input.

As shared by Teresa Torres in Continuous Discovery Habits, using insights in product decision-making has many benefits:

1. Better alignment. 
   
   Team members know why they’re working on something, and have a shared understanding of what users care about.
2. Evidence-based prioritization.
   
   Teams prioritize based on insights they’ve gathered over time. Deciding where to invest becomes much more objective.
3. Easier roadmapping.
   
   Clarifying investment areas becomes straighforward, and nsights gathered over time provide weight to potential areas of investment. Wise resource allocation becomes clear and trade-offs can be made in an informed way.
4. Better stakeholder engagement.
   
   A clear narrative about customer needs and priorities is self-evident. Teams easily communicate this narrative to stakeholders, reframing conversations around customer struggles and impact.

Discussing insights at Jira Product Discovery

On the Jira Product Discovery team, we use two rituals to make continuous discovery a habit:

• Weekly “feedback rotation” assignment: 

Each week, a person on the product team is assigned to “feedback rotation.” They look at our channels, respond to feedback and add relevant Insights to Ideas. This usually takes about a half day of effort, spread through the week.

• Weekly “data bath” meeting:

In this team meeting, we go over that week’s feedback to discuss learnings and takeaways. At the end of that meeting, we officially pass over the “feedback rotation” assignent to the next product manager.

Because we’ve done this work on an ongoing basis, we’re prepared for all prioritization discussions with plenty of fresh context.

If you receive feedback from many customers, who should you listen to?

At Atlassian, we have more than 300,000 customers. On the Jira Product Discovery team alone, we receive dozens of individual pieces of feedback about the product every week. Working with all their feedback directly, let alone acting on it, would be impossible.

Inbound user feedback gives us a sense for the types of issues people face when using the product, and which ones are more pervasive. This feedback informs product Ideas, but we don’t design solutions based on that feedback alone.

Instead, as we develop, iterate, and ship solutions, we typically work with a small, dedicated group of users. We build product experiences with and for them, listening to their highly detailed, targeted feedback. 

We call these test groups “lighthouse users”. In our experience, 10 lighthouse users is enough.

Why work with lighthouse users?

In our experience, working with lighthouse users works better than trying to please thousands of users all at once.

• We move faster, because we can make decisions based on discussions with a small number of people.
  
  We don’t need to survey 1000 users to get to a decision. We can test product experiences early by giving lighthouse users prototype access before anyone else. These users know what to expect: there may be bugs and corner cases that are not covered.
• We get rich contextual insights, because we’ve built rapport with these users to understand their problems and motivations.
  
  It’s often more impactful to have multiple conversations with a user you know well, than to have a single conversation with many people.
• It drives urgency across the whole team, because they know our lighthouse users as people and want to help.
  
  Introducing lighthouse users to the whole team drives their sense of urgency and ownership. Solving problems for one person they care about is much more motivating than a research report.

How to select lighthouse users

If you decide to adopt this way of working, you must be very clear on who is a good fit as lighthouse users. If you choose wrong, you could end us building the wrong product for the wrong people. 

Over time at Jira Product Discovery, we’ve found that our lighthouse users share the following attributes:

✅ Very clear communicators

✅ In our target customer segment, even if we’re still iterating on what that is

✅ Strongly affected by the problems and pain points we’re trying to solve

✅ Trying to find new ways to solve this pain point, so far without success

✅ Not using a competing app, so we don’t center discussions on replicating features

✅ Open to new and different ways of working, instead of looking for specific features 

✅ Happy to use an early-stage product or feature, share feedback, and discuss solutions

Working with lighthouse users

We really go all in to collaborate with lighthouse users. Essentially, we treat them as co-creators of the solution.

• We set up dedicated Slack channels where they can share feedback and questions
• We meet monthly to discuss their product, and product management, experiences
• We share early designs with them and get their feedback
• We give them early access to products, experiences, and features 
• We ask how they’d design the solution themselves

For more information about lighthouse users, see this series of posts on the Atlassian Community.

What’s next?

By grounding all their decisions in Insights, product teams can keep their work focused on its ultimate goal — solving users’ problems and making their lives easier.

In the final two sections of this handbook, we’ll explain in detail how to use a product backlog to:

• Prioritize the ideas that can make an impact
• Create roadmaps your teams and stakeholders can rally behind

We’ll give examples for how we do this in the Jira Product Discovery team, using Jira Product Discovery and other products.