Advanced Reviews Deliver User Insights at Scale

Today, App Annie launches Advanced Reviews in our Intelligence solution. Advanced Reviews accomplish two critical things for app product managers and app marketers. First, it enables them to quickly surface insights at scale, without any manual work. Second, it enables them to understand competitor reviews to improve performance and user sentiment. 

What’s powering these insights? Natural Language Processing (NLP), which App Annie has applied to the app store. The NLP machine learning algorithm behind Advanced Reviews reads, deciphers, and understands human language at scale to help you find and fix user-reported app issues. The algorithm can surface issues early on, so that you have time to respond and correct them before they have a significant negative impact on ratings and downloads.

Consider the example of the Domino’s app. With over 3.7 million ratings and reviews, Domino’s Pizza can teach apps a thing or two about keeping users happy and driving downloads. The Domino’s iOS app is among the most successful quick service restaurant apps around. Developers can learn a lot from these reviews, but who has time to sift through 3.7 million comments? While most reviews might be relevant to the core business of pizza, some reviews will reflect the app’s performance and the quality of the user experience. This is extremely important. Missing those reviews might mean developers are missing opportunities to catch bugs early and double down on wins. For example, learning that the latest version of the app crashes right before entering payment information could increase revenue and keep pizza lovers from tapping over to a competitor with better app performance.  

Instead of leaving this to manual processes that might leave countless app users pizza-less  until a QA team finally finds the bug, this is where App Annie’s Advanced Reviews come in. The NLP algorithm addresses and categorizes all 3.6 million reviews in a single afternoon. Without it, a QA team might spend a week or more and produce inferior results filled with reviewer biases.

In this example, you can see that top phrases like "sign in" are driving low average ratings, resulting in a negative impact score of minus ten. This experience is mentioned in over 65% of reviews.

Reviews Are Key to App Store Performance 

High quality reviews and ratings significantly impact whether an app is featured in the app stores, since the app stores regularly seek to promote apps that provide excellent consumer experiences. A portfolio of great apps validates the role of the app store and the platform itself. For a publisher, getting featured drives download conversions. It is a cycle that helps high quality apps drive even more downloads and increase satisfied users over time. 

The opposite is also true. Apps that provide a poor experience decrease the likelihood of getting featured in the app stores and will subsequently suffer lower download conversions. 

Improve QA Efficiency 

Reviews are a goldmine for information on bugs and other performance issues. Solely relying on a QA team to surface all bugs is a costly endeavor, both in terms of time and money. Without Advanced Reviews, iOS app publishers are stuck with a manual process that is unable to keep up with the fast-paced nature of the app market. 

Protect Market Share from Competitors

How well-versed are you in your competitors’ customer satisfaction scores? Reviews provide an unparalleled window into how competitors are meeting user needs. In addition, competitor responsiveness to reviews (or their lack thereof) is worth comparing to your own. After all, a quick developer response to a critical review, explaining what you have implemented to address their concern, can quickly turn a 1-star rating into a 5-star rating and gain you a loyal lifetime user. If you are unable to respond to critical reviews and fix the bugs they mention, you give up market share to competitors that easily provide users the experience they seek. Think of any popular podcast app. If a user’s favorite podcast simply won’t play, they have an incentive to download another so that they can listen successfully. You want to find out sooner rather than later about any app issues that could potentially push your users to a competitor and address them quickly. 

With Topic Analysis, you can see trends in topics and identify which topics are raised most often.

How Do Advanced Reviews Work? 

Advanced Reviews employ two Natural Language Processing workflows to glean insights from reviews data. 

The first is Topic Classification - each review is tagged with one or more topic classes which range from “performance and bugs” though “service rendered” and “design and interface.” This provides app developers with a snapshot of the topics their users are discussing in reviews and how these evolve over time. For example, if the proportion of reviews tagged as “performance and bugs” shoots up immediately after an update, that might be a signal that the update is buggy and needs to be reviewed.

Topic classification is a difficult problem because it tries to capture the myriad of features that humans use when deciding how to classify a review. To achieve it, we use a powerful transfer learning algorithm, which is trained to recognize review topics using a large, pre-classified dataset that we provide. The results mimic human-level topic classification performance, and they speak for themselves in our product.

The second NLP workflow we use is called Relevant Terms. Here, we use a proprietary algorithm that was developed in-house to extract key phrases from reviews and assign each phrase an “impact score,” which is designed to quantify both the importance and sentiment of that phrase within the selected group of reviews. This process is designed to extract readable, actionable phrases from the text and then rank them according to priority to address. Within our product, if the user clicks on a relevant term, they are taken to the reviews that contain that term. Thus, in just a few clicks, they can read and learn from these reviews that contain critical information about their app and their users’ satisfaction.

Secure Future Success with Advanced Reviews

Planning a robust app roadmap often requires the timely and costly process of surveying current users. Advanced Reviews give you the data to validate your roadmap and eliminate the guesswork by focusing on features many users have already requested. Because with every review, your users are telling you what they expect. Tapping into their feedback gives you a pre-vetted and de-risked path forward — leading to new features for which your users have already asked.

The Reviews Summary shows critical review information, with the ability to use filters to hone in on key areas.

How Advanced Reviews Fit Into a Comprehensive Mobile Strategy 

The value that app publishers can glean from Advanced Reviews extends beyond user sentiment and fits in with App Annie’s robust Marketing Intelligence offerings. Reviews can even uncover surprising and vital information, such as new keywords reviewers are using to define your app. These recurring keywords can roll into your comprehensive ASO strategy with the help of App Annie’s marketing intelligence suite — increasing your app store visibility and improving download conversions. 

The insights you can gain from Advanced Reviews — and the resources you can save — are unmatched. From bug detection to a full competitive understanding, Advanced Reviews enable app publishers to get deep into the reviews and ratings of any app on the market to find action items in less time. 

Get in touch with App Annie today to learn how you can boost your performance and user satisfaction in the app stores with Advanced Reviews.