11436 SSO

Analytics Platforms: Data Capture, Data Processing, Data Visualization

Prabhat Jha
Feb 01, 2013

We recently blogged about different kinds of mobile analytics and categorized them as market, operational and customer experience analytics.

Whatever your analytics needs, analytics systems typically have three parts: data capture, data processing and data visualization. Let’s examine each of these three in the context of mobile analytics how they drive the requirements of a mobile analytics solution.

Data Capture

For a mobile app, whether it’s native, Web, or hybrid, analytics data capture is generally done through an SDK. Unlike server side applications where you control your deployment platform (OS, Middleware, network configuration, etc.), mobile apps run in environments where the permutation of platforms (Android, iOS . . .), platform versions, device models, network types (3G, 4G, WIFI . . .), network carriers (AT&T, Verizon, Sprint, T-Mobile . . .), though finite, quickly become unmanageable and makes it practically impossible to test every combination.

These variations make the development of a generic SDK difficult because an SDK should be minimalistic, fail safe, and have a consistent API to the extent allowed by the underlying platforms. An app should be able to function even if an SDK does not have everything it needs. For example, if an app does not have permission to capture location, a generic SDK should proceed gracefully. In other words, a failure in the SDK should not cause a failure in the app.

Consider also that mobile devices may not always be connected. An SDK should handle the scenario in which devices are not connected at all or get temporarily connected during a session but cannot send captured data.

My colleague, Paul Dardeau, recently built this great list - Developer Expectations for 3rd Party SDKs/Libraries/Frameworks. We believe that every good mobile SDK should strive to follow these 42 considerations.

Data Processing

Once you have data, the processing of that data (whether for app performance or customer experience analytics) is similar to any other data analytics system with some caveats for mobile.

First, a device may not have connectivity all the time. It is a very likely scenario that SDK sends data at the start of a session but is not able to do so for the rest of the session or vice versa. This skews the session length and unique session count calculation, especially for near real-time analytics. The data processing system needs to be able to access older data and update already calculated analytics. In other words, if an app has been in use for a while but is not able to send the data for a longer period of time, the SDK needs to drop some “stale” data to conserve the data limit and the processing engine needs to gracefully handle “incomplete” data with interpolation or extrapolation.

Second, the data volume can vary significantly. A self service generic mobile analytics data processing system needs to efficiently handle data from apps with hundreds of downloads as well as from those with millions of downloads. In other words, the system needs to be highly elastic with respect to load.

Data Visualization

A lot of existing server-side business analytics systems or application performance analytics solutions claim to have the same for mobile apps. I believe that’s putting lipstick on a pig. An application deployed on the server side has one hardware configuration, one software configuration, one network configuration, and one deployment policy.

Mobile apps however can have network connectivity issues and varying data volumes depending on the usage of the app (as explained above) so data visualization becomes inherently different for them.

A performance dashboard for mobile apps should not only show where an app is not working but should have the ability to efficiently drill down by device platform, platform version, device model. Similarly, when analyzing network performance, a system should provide drill down by network type and network carrier.

We’ve built our Mobile Analytics solution from the ground up with these data capture, processing and visualization needs in mind.

If you have comments, questions, or need help optimizing your mobile app, feel free to reach out to us at mobile@apigee.com.

Scaling Microservices