Common Problems In Product Analytics

Vikram Aditya
CEO & Co-founder
July 3, 2023

While there are thousands of SaaS products out there and a lot of them have absolute substitutes, there is not a single team of product builders that's absolutely the same, hunting for the same metrics, and trying to answer the same business queries. With that in mind, it doesn’t matter how many product analytics tools there are in the world, we are always looking to solve one or the other problem statement. 

From Data Dilemmas to Actionable Insights, there are still many problems that are persistently faced by PMs and analysts then it comes to analyzing their product’s usage, despite the rapid proliferation of new analytics tools. We explore some of these problems.

1. OPTIMALLY LEVERAGING DATA WITH A LOWER LEARNING CURV

Data compilation is just the first step, making sense of the collected data is what matters the most. While there are analytics platforms (albeit not too many), that are phenomenal when it comes to the collection of data but doesn’t help much in making sense of it. Probably because of problems like an unintuitive UI/UX for dashboard setup, irrelevant integrations, and data tracking/filtering options.

Modern day LLMs are a breakthrough for this problem, because the user can engineer the prompt as per their requirements and generate the user data in a way that’s of most value to them and their role within the organization. Moreover, they can do this using natural language.

The significance of being able set up mechanisms to handle your data in English vs code cannot be understated: 

While to seasoned analysts it may seem like a mere nice to have, the impact of such accessibility is that it drastically decreases how easily others on a team can leverage data. For example, your. marketing team may well benefit greatly from knowing and using product data better. However, to do so, they'd likely have to go through someone on the team well acquainted with usually complex analytics tools. Using Natural Language for the same solves this problem.

3. COMPLICATED AND INTIMIDATING UI/UX

Similar to and intertwined with the aforementioned factor, the idea of creating another dashboard or tracking a new event/metric might come from the product team or the business analysts or the CS teams, but the tagging and designing part of the dashboard creation is done by engineers and PMs.This tends to result in a dashboard that just might be easy to comprehend for the product team, but not so much for the CS teams. As a result, this creates a massive gulf in communication and alignment within the teams.

What we are also looking at here is not just how the dashboards can be complex, but also how the same data from the same source can be perceived differently by different teams within an organization.

With generative AI exponentially advancing, it shall become even easier for all the teams to perceive the data consonantly since it’ll be portrayed in a rather plain language.

4. IMPROPER DATA GOVERNANCE

This leads to dashboards and charts that are inaccurate. With most product analytics platforms needing you to painstakingly tag events, it is even more important to have appropriate data governance in place to ensure there are no discrepancies when the dashboard is finally live.

A lot of the success comes down to your organization's implementation of the front-end events. Some time spent upfront on event strategy and implementation is essential to really get the most out of most product analytics tools. Data governance is key here.

Unless you are 100% certain it is set up properly, the data does not and frankly, should not, engender trust.

A viable solution for this problem would be to opt for an autotracking enabled analytics solution that tracks data intelligently, understanding what all the events actually are.

5. FILTERING OUT YOUR OWN EMPLOYEES IN USER ANALYSIS

Especially for products that are just starting out, not being able to distinguish 'real' usage from your own team members engaging with the product for developmental purposes is quite an issue.

For example, if your product has just launched and there are 100 engineers working on it daily, the data derived will be highly inaccurate, with false events sometimes even going into the thousands.

Now, despite there being several solutions to circumvent this problem, they don't come without problems of their own: 

  • Blocking Company Emails: Some users might be testing the platform using their alternate email IDs
  • IP Blocking: Untenable in any remote work setup
  • Blocking using device IDs: Efficient and viable, but scarcely available in most platforms

6. INEFFECTIVE DEFAULT EVENT TRACKING

DET, now available with many analytics platforms,tends to be quite substandard, and as a result, provides paltry value.

For example, the auto-tracking of 15-20 events by default definitely saves time, but it still requires the PMs and engineers to set up tagging for other, more crucial events. Furthermore, if you already have a dashboard setup then it doesn’t really affect you much.

Given how useful DET can really be, removing the downsides that currently come with it would be a game-changer.

Using a purpose built, fine-tuned model, this is now possible.

Now, we dive deeper into the experiences and insights shared with us by industry professionals through one-on-one interviews. From dashboard creation to data accuracy, let's explore the key challenges faced by PMs and analysts in their day to days workflows.

7. CHALLENGES MENTIONED BY PMs

Dashboard Creation & Usage

Creating effective and informative dashboards is a crucial aspect of product analytics. However, our interviews revealed various challenges related to dashboard creation, updates, and distribution. The time and effort required to set up events, track them, and design intuitive dashboards can be significant.

Moreover, keeping dashboards up-to-date can be a demanding task, requiring continuous monitoring and updates. Additionally, sharing dashboards with teammates and ensuring they are as accessible to all teammates is near impossible, given the varying degrees of expertise across verticals in a company. As of yet, there's not really been a solution for this.

However, by being able to make dashboards almost an order of magnitude more quickly (that too using plaintext) enabels people across your team to do so. This enables dashboards to be created for hyper specific levels of expertise and use cases.

Data Consistency & Accuracy

Achieving data consistency and accuracy is a recurring challenge faced in product analytics. Ensuring that data is collected accurately across different tools and platforms can be complex, especially when inconsistencies arise. Factors such as ad blockers and data discrepancies across various analytics tools can hinder the accuracy of insights derived from the data. Resolving these issues often requires meticulous troubleshooting and rectification efforts.

As of yet, there’s been no viable means to accomplish this.  However, training purpose-built LLMs to disregard anomalies in the data collected are now tenable. Such a model would be capable of removing any and all data points that were adulterated because of ad blockers, sync issues, technical glitches, etc.

Gleaning Insights & Deliberation

Perhaps the most crucial challenge. Extracting valuable insights from the data and using them for effective decision-making is a primary goal of product analytics. Our interviews revealed that industry professionals focus on metrics such as user behavior, segmentation, and funnel analysis to gain actionable insights. Ctreating the right type of dashboard and maintaining it with every new feature update can be a tedious, and more importantly, error prone task.

An AI model in such a case can be a breakthrough by being able to suggest the types of charts that should be created to monitor a particular set of business questions.

As organizations increasingly rely on data-driven strategies, the challenges encountered in product analytics become more significant. From creating comprehensive dashboards to ensuring data accuracy, industry professionals face a variety of obstacles on their path to deriving actionable insights. The experiences shared by the interviewees emphasized the importance of continuous tooling evolution, learning, and a good understanding of the organization's specific needs. By addressing these challenges head-on, businesses can unlock the true potential of product analytics and make data-informed decisions that drive success in today's competitive landscape.

At Crunch, we're solving every problem spoken about here by weaving modern AI with analytics.

Sign up for our waitlist here!


Ready to get valuable Product Insights?

If you are looking to make better use of your product data, gain insights faster and improve decision making across teams, Crunch can help you get there.
Book a demo
July 3, 2023
Development

Common Problems In Product Analytics

From Data Dilemmas to Actionable Insights, there are still many problems that are persistently faced by PMs and Analysts then it comes to analyzing their product’s usage, despite the rapid proliferation of new analytics tools. We explore some of these problems.

Common Problems In Product Analytics

While there are thousands of SaaS products out there and a lot of them have absolute substitutes, there is not a single team of product builders that's absolutely the same, hunting for the same metrics, and trying to answer the same business queries. With that in mind, it doesn’t matter how many product analytics tools there are in the world, we are always looking to solve one or the other problem statement. 

From Data Dilemmas to Actionable Insights, there are still many problems that are persistently faced by PMs and analysts then it comes to analyzing their product’s usage, despite the rapid proliferation of new analytics tools. We explore some of these problems.

1. OPTIMALLY LEVERAGING DATA WITH A LOWER LEARNING CURV

Data compilation is just the first step, making sense of the collected data is what matters the most. While there are analytics platforms (albeit not too many), that are phenomenal when it comes to the collection of data but doesn’t help much in making sense of it. Probably because of problems like an unintuitive UI/UX for dashboard setup, irrelevant integrations, and data tracking/filtering options.

Modern day LLMs are a breakthrough for this problem, because the user can engineer the prompt as per their requirements and generate the user data in a way that’s of most value to them and their role within the organization. Moreover, they can do this using natural language.

The significance of being able set up mechanisms to handle your data in English vs code cannot be understated: 

While to seasoned analysts it may seem like a mere nice to have, the impact of such accessibility is that it drastically decreases how easily others on a team can leverage data. For example, your. marketing team may well benefit greatly from knowing and using product data better. However, to do so, they'd likely have to go through someone on the team well acquainted with usually complex analytics tools. Using Natural Language for the same solves this problem.

3. COMPLICATED AND INTIMIDATING UI/UX

Similar to and intertwined with the aforementioned factor, the idea of creating another dashboard or tracking a new event/metric might come from the product team or the business analysts or the CS teams, but the tagging and designing part of the dashboard creation is done by engineers and PMs.This tends to result in a dashboard that just might be easy to comprehend for the product team, but not so much for the CS teams. As a result, this creates a massive gulf in communication and alignment within the teams.

What we are also looking at here is not just how the dashboards can be complex, but also how the same data from the same source can be perceived differently by different teams within an organization.

With generative AI exponentially advancing, it shall become even easier for all the teams to perceive the data consonantly since it’ll be portrayed in a rather plain language.

4. IMPROPER DATA GOVERNANCE

This leads to dashboards and charts that are inaccurate. With most product analytics platforms needing you to painstakingly tag events, it is even more important to have appropriate data governance in place to ensure there are no discrepancies when the dashboard is finally live.

A lot of the success comes down to your organization's implementation of the front-end events. Some time spent upfront on event strategy and implementation is essential to really get the most out of most product analytics tools. Data governance is key here.

Unless you are 100% certain it is set up properly, the data does not and frankly, should not, engender trust.

A viable solution for this problem would be to opt for an autotracking enabled analytics solution that tracks data intelligently, understanding what all the events actually are.

5. FILTERING OUT YOUR OWN EMPLOYEES IN USER ANALYSIS

Especially for products that are just starting out, not being able to distinguish 'real' usage from your own team members engaging with the product for developmental purposes is quite an issue.

For example, if your product has just launched and there are 100 engineers working on it daily, the data derived will be highly inaccurate, with false events sometimes even going into the thousands.

Now, despite there being several solutions to circumvent this problem, they don't come without problems of their own: 

  • Blocking Company Emails: Some users might be testing the platform using their alternate email IDs
  • IP Blocking: Untenable in any remote work setup
  • Blocking using device IDs: Efficient and viable, but scarcely available in most platforms

6. INEFFECTIVE DEFAULT EVENT TRACKING

DET, now available with many analytics platforms,tends to be quite substandard, and as a result, provides paltry value.

For example, the auto-tracking of 15-20 events by default definitely saves time, but it still requires the PMs and engineers to set up tagging for other, more crucial events. Furthermore, if you already have a dashboard setup then it doesn’t really affect you much.

Given how useful DET can really be, removing the downsides that currently come with it would be a game-changer.

Using a purpose built, fine-tuned model, this is now possible.

Now, we dive deeper into the experiences and insights shared with us by industry professionals through one-on-one interviews. From dashboard creation to data accuracy, let's explore the key challenges faced by PMs and analysts in their day to days workflows.

7. CHALLENGES MENTIONED BY PMs

Dashboard Creation & Usage

Creating effective and informative dashboards is a crucial aspect of product analytics. However, our interviews revealed various challenges related to dashboard creation, updates, and distribution. The time and effort required to set up events, track them, and design intuitive dashboards can be significant.

Moreover, keeping dashboards up-to-date can be a demanding task, requiring continuous monitoring and updates. Additionally, sharing dashboards with teammates and ensuring they are as accessible to all teammates is near impossible, given the varying degrees of expertise across verticals in a company. As of yet, there's not really been a solution for this.

However, by being able to make dashboards almost an order of magnitude more quickly (that too using plaintext) enabels people across your team to do so. This enables dashboards to be created for hyper specific levels of expertise and use cases.

Data Consistency & Accuracy

Achieving data consistency and accuracy is a recurring challenge faced in product analytics. Ensuring that data is collected accurately across different tools and platforms can be complex, especially when inconsistencies arise. Factors such as ad blockers and data discrepancies across various analytics tools can hinder the accuracy of insights derived from the data. Resolving these issues often requires meticulous troubleshooting and rectification efforts.

As of yet, there’s been no viable means to accomplish this.  However, training purpose-built LLMs to disregard anomalies in the data collected are now tenable. Such a model would be capable of removing any and all data points that were adulterated because of ad blockers, sync issues, technical glitches, etc.

Gleaning Insights & Deliberation

Perhaps the most crucial challenge. Extracting valuable insights from the data and using them for effective decision-making is a primary goal of product analytics. Our interviews revealed that industry professionals focus on metrics such as user behavior, segmentation, and funnel analysis to gain actionable insights. Ctreating the right type of dashboard and maintaining it with every new feature update can be a tedious, and more importantly, error prone task.

An AI model in such a case can be a breakthrough by being able to suggest the types of charts that should be created to monitor a particular set of business questions.

As organizations increasingly rely on data-driven strategies, the challenges encountered in product analytics become more significant. From creating comprehensive dashboards to ensuring data accuracy, industry professionals face a variety of obstacles on their path to deriving actionable insights. The experiences shared by the interviewees emphasized the importance of continuous tooling evolution, learning, and a good understanding of the organization's specific needs. By addressing these challenges head-on, businesses can unlock the true potential of product analytics and make data-informed decisions that drive success in today's competitive landscape.

At Crunch, we're solving every problem spoken about here by weaving modern AI with analytics.

Sign up for our waitlist here!


Customer retention is the key

Lorem ipsum dolor sit amet, consectetur adipiscing elit lobortis arcu enim urna adipiscing praesent velit viverra sit semper lorem eu cursus vel hendrerit elementum morbi curabitur etiam nibh justo, lorem aliquet donec sed sit mi dignissim at ante massa mattis.

  1. Neque sodales ut etiam sit amet nisl purus non tellus orci ac auctor
  2. Adipiscing elit ut aliquam purus sit amet viverra suspendisse potent
  3. Mauris commodo quis imperdiet massa tincidunt nunc pulvinar
  4. Excepteur sint occaecat cupidatat non proident sunt in culpa qui officia

What are the most relevant factors to consider?

Vitae congue eu consequat ac felis placerat vestibulum lectus mauris ultrices cursus sit amet dictum sit amet justo donec enim diam porttitor lacus luctus accumsan tortor posuere praesent tristique magna sit amet purus gravida quis blandit turpis.

Odio facilisis mauris sit amet massa vitae tortor.

Don’t overspend on growth marketing without good retention rates

At risus viverra adipiscing at in tellus integer feugiat nisl pretium fusce id velit ut tortor sagittis orci a scelerisque purus semper eget at lectus urna duis convallis porta nibh venenatis cras sed felis eget neque laoreet suspendisse interdum consectetur libero id faucibus nisl donec pretium vulputate sapien nec sagittis aliquam nunc lobortis mattis aliquam faucibus purus in.

  • Neque sodales ut etiam sit amet nisl purus non tellus orci ac auctor
  • Adipiscing elit ut aliquam purus sit amet viverra suspendisse potenti
  • Mauris commodo quis imperdiet massa tincidunt nunc pulvinar
  • Adipiscing elit ut aliquam purus sit amet viverra suspendisse potenti
What’s the ideal customer retention rate?

Nisi quis eleifend quam adipiscing vitae aliquet bibendum enim facilisis gravida neque euismod in pellentesque massa placerat volutpat lacus laoreet non curabitur gravida odio aenean sed adipiscing diam donec adipiscing tristique risus amet est placerat in egestas erat.

“Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua enim ad minim veniam.”
Next steps to increase your customer retention

Eget lorem dolor sed viverra ipsum nunc aliquet bibendum felis donec et odio pellentesque diam volutpat commodo sed egestas aliquam sem fringilla ut morbi tincidunt augue interdum velit euismod eu tincidunt tortor aliquam nulla facilisi aenean sed adipiscing diam donec adipiscing ut lectus arcu bibendum at varius vel pharetra nibh venenatis cras sed felis eget.

Subscribe to our newsletter today!

Case studies and play-books on the best digital experience builders

Thanks for subscribing to our newsletter
Oops! Something went wrong while submitting the form.
Subscribe to Our Newsletter - Marketup X Webflow Template