With today's rapid adoption, mobile apps economy, and open APIs, understanding how to measure an API program is less predictable than it once was, and less predictable than when you are working internally or with partners.
What are good metrics to measure an API program?
The best way to approach the question of metrics is from the perspective of who is going to act on them. Successful API teams understand the value of people and technology and two sets of people make APIs successful: app developers and the API team. App developers are innovating your API and building creative apps. The API team is tapping into internal systems to create interesting APIs.
What you measure is based on what is actionable by the three types of people who make your API business successful: app developers, the API team, and the operations team.
Apigee offers charts and data that allow you to manage performance and API health. See Monitor the performance of your API for more information.
What does the API team need to measure?
Almost everything!
The API team needs visibility into API operations and business. Here are some of the things that help you really understand your users so you know where to invest for the future:
- What are our top apps? You need to know who/which apps are using your API.
- Who are our top app users?
- Who are our best app developers? Your best users and developers might be operating with you across multiple apps and devices, such as website, mobile apps, video game consoles, and tablets.
- Which API methods are most popular?
Your API is a product; you want to understand its usage to help pivot your business and make future investment decisions. - How much API capacity will be needed next year?
Capacity planning is difficult. What if your API goes viral and you have millions of users? You need trending usage data to help make these decisions. In this context, you want to see indicators in the trend charts like the number of requests (the load hitting the back-end systems), volume of data (especially for API responses), and so on.
When things are not going so well, the API team needs to know these things:
- Why is the API down?
There’s probably not one thing to tell you why an API is down, but the shape of your graphs can help. Do your charts show catastrophic failure or a slow lead-in to failure? - Why is the API slow?
You can do simulated transactions; look across API methods and also across apps. Could it be that apps are doing something that’s slowing your API? - Why is the API throwing errors?
- Why is API traffic spiking?
Is something new going on? Can you nail down the source of traffic — for instance, has a new app come online? - Why did the API traffic disappear?
This is the opposite of spiking traffic. Do you have a business issue, such as did everyone uninstall the app overnight? Or is it a back-end system problem not necessarily to do with the software, such as you forgot to renew certificates or DNS?
A mix of technology and people make your API business successful, so here are the key indicators your API team looks for:
- App users
- Apps
- Developers
- API quality
- Internal systems data (database query times, message bus response times, external call-out response times, etc.)
In addition to knowing how app developers, apps, and their end users are using the API, the API team needs to measure what’s going on behind the API in the internal (back-end) systems.
Internal system data can be held in response headers. Apigee can log this data in the database with custom policies and provide metrics to the API team/provider and even to the app developer if appropriate.
Apigee offers charts and data that allow you to manage performance and API health. See Monitor the performance of your API for more information.
You can also build custom reports. See You do not have access to view this node.
What does the app developer need to measure?
Some important questions that metrics can answer for app developers are:
- Who is their app user?
- How are apps performing?
- From a customer perspective, how is the API working?
Drilling a little deeper, app developers might have several general questions about the API. Use trending charts can help answer these questions:
- Is the API error prone?
- How does the API usually perform?
- How often is the API unavailable?
There are also questions that are more specific to the app developer or app's usage of the API or to the point in time:
- Which API errors is my app seeing?
- Is the API slow right now?
- Which API methods are typically slow or slow right now?
- Does your API have a quota? And as the app developer, how am I doing against the quota? Am I violating the quota?
You don't want your app developers to waste time debugging software that doesn’t have a problem, so knowing that there is a quota and how their apps are doing against that quota is important. - Is the API down right now?
- When will the API be back up?
- Why was the API down?
Data and charts in the developer portal can help developers answer these questions, improve their apps, and plan for the future.
Note: This is different data than that needed by the API team.
Here are some key indicators for developers.
- Errors: Rate, code, and descriptions
- Performance: Trend graphs for the API (all data) and for the developer's specific apps using the API
- Availability: Whether the API up or down
- Quota: If enforced, quota limits and how an app is doing against them
