Developer Spotlight: How CalorieCount, part of About.com, is using Apigee to eliminate 1-star app reviews
Calorie Count's app is a free health and wellness tool that allows mobile users to search and log nutritional data for over 300K foods, popular exercises and healthy recipes, as well as track their weight loss, all while on the go on your mobile devices. The Calorie Count app is an extension to CalorieCount.com, an award-winning free online health and wellness service with almost 5.6 million members.
We asked Igor Lenovic, VP at About.com about his experience analyzing the Calorie Count mobile app behavior and performance using Apigee Mobile Analytics.
What metrics do you use to measure the success of your mobile app?
In the hyper-competitive world of fitness apps, customer ratings of your app is crucial. The number 1 metric we use is the ratio of 1 stars to 5 stars. We currently get 2 1-star ratings for every 100 5-star ratings. Our goal is to get that ratio down to 1% . The ratio is a much better indicator of customer satisfaction than “ average rating” .
What challenges did you face getting to that goal?
We had a few issues to address. We wanted to improve our overall instrumentation of our app. You can’t improve what you cannot measure - right? We needed to improve our instrumentation so that we could identify the root causes of our 1 star reviews and eliminate them.
Then we needed to address two pretty specific problems. First, an interaction between the iOS client and server would occasionally cause the app to fail, but not crash. Second, we were seeing a spike in app crashes once per week on the same day.
What instrumentation did you have?
We were using a crash detection software. However, it only caught crashes and not application errors or network errors. We also used a large host of Google Analytics custom events to track app errors, but it was really a hack.
We looked at Apigee’s solution and found it to be much more “elegant”. Apigee Mobile Analytics goes beyond crash and error reports, and enables us to go beyond what we were able to do with Google Analytics. We can pinpoint performance issues in the app easily and in real time using network and log analytics.
Describe the issue with the interaction between the iOS client and server?
An interaction between the iOS client and server would occasionally cause the app to fail, but not crash. It was a problem that spanned the API and the app, and there was no instrumentation that could diagnose the root cause.
So we needed something that covered both the API and the app.
Hal, our server-side guru needed instrumentation at the app layer to determine if the problem was at the server or at the app. Specifically, we needed to capture the logging events that was generated by the app, which devices the errors occurred on, and the corresponding API calls that were made - Apigee’s SDK provides us this capability to track all three criteria.
Describe the issue with the spike in weekly crashes
We were getting a minor spike in crashes every Thursday. However, Thursdays corresponded to our lowest traffic of the week. It was a head scratcher for our developers, and we didn’t have the analytics to diagnose it. Our crash detection software indicated only that there was a small spike in crashes on Thursday, but didn’t indicate where to begin looking.
That’s like finding a needle in a haystack. How did Apigee help solve the problem?
Your data scientist will have the answer that question in detail (see next blog post). All I know is that he managed to isolate the exact location in crashes from the data we already had. Our developer suspects that the data that was being sent back from our API was causing the app to crash.
It seems that the interaction between API and app was the cause again.
So what else happened?
When we instrumented our Android app, our developer discovered that there was extremely high latency on particular API calls. Apigee’s network analytics has enabled Hal, our API guru, to collaboratively debug these issues with our app developer. Apigee’s tools facilitated this collaboration.
What’s your advice for others with similar challenges? Would you do anything differently?
Again, what you can’t measure, you can’t improve so I’d say instrument everything.
It was also critical for us to have the support of Apigee data scientists to analyze our existing data.
With 1-star ratings, it’s likely to be a combo of the app and API that is causing issues, but you don’t know until you can measure and do solid root cause analysis.
Using Apigee Mobile Analytics and with the support of Apigee data scientists, we were able to diagnose and ﬁx our Thursday crash spike issue. We expect to see our star ratings ratio improve as a result of the fixes we’ve made very soon.
Great! What’s next for CalorieCount and Apigee?
We’ve still got to solve the issue with the interaction between the iOS client and server. Also, our development team is looking at a new UI Event tracking mechanism that your Mobile Analytics team is developing. We’d also like start trying out the configuration management and A/B testing capabilities.