11436 SSO

Insights from API data - visibility across the value chain

anant
Mar 07, 2012

Thanks to all who participated in last week's Webinar: Visibility at the Edge - Deep Insights from your API. The video and slides are here.

As promised on the Webinar, we'll write out some of the Q&A we got into in the live session. We'll start with what we mean by visibility at the edge and why we think it's important.

What do you mean by visibility at the edge?
 
in the past, enterprises could focus on controlling their interactions with customers through physical presence or websites. Now people are interacting more and more with core businesses through apps. 
Although the interaction has moved farther away and businesses are no longer in control of the interaction in the same way, customers are more important to the business than ever before. More data is coming in from developers and end users through apps and APIs.
 
Clearly, to get such visibility, we will collect lots of data. Before we get into the data part, can you give examples of what such visibility might tell us?
 
Think about the number of different constituents around API interactions (even non-API interactions). There are the people transacting with your business, developers building apps, teams in enterprises responsible for the building and the success of APIs, teams responsible for the operations of APIs, and business users interested in the top and bottom lines. 
 
<<image>>
There are also lots of interactions through various channels with different systems, including billing, catalog, procurement, order management, and so on. The biggest challenge is how to collect the disparate perspectives you get through these channels and generate a 360 degree sense of it. If you believe in the apps economy, you believe this will all get standardized around the APIs "interface."
 
Your APIs and the management of APIs gives you visibility into a larger set of interactions than any one back-end system can. If you harness that visibility, regardless whether you run operations for the back-end systems, your business is to make the APIs successful, or you are a business person focused on the top and bottom line, you can get a lot of insights into your business. For example: 
 
- How well is the API being adopted?
- What is the response time per request?
- Does operations need to allocate more resources at peak times?
- What impact is an API having on my bottom line?
 
But there's something even more transformative one can get from API data. Instead of thinking only about what the enterprise learns from API data, think about what developers and the app end users can learn from the API data. For example, if you are a developer, you'd like to know what impact you're having on the business whose API you are using to build your app. 
 
Visibility into the APIs gives insights to all of the constituents, therefore providing a 360 and actionable view.
 
 
What’s needed to make this 360 degree visibility happen?
 
There are a few things. First obviously, you need to be collecting data from the APIs. 
Second is collecting data that adds context to the API data. 
Thirdly, you need analytics that model and predict both the business and operational metrics.
 
Let's elaborate on data that adds context.
 
<<360 image>
 
Visibility into APIs is the best aperture into enterprises but just API data is insufficient - you need a mechanism to collect other "contextual" data. 
 
Say a user is interacting with a Telco API. A cell phone number is flowing through that API. Other than that phone number, there's a huge amount of context that is not captured in that API:
- who's number is it?
- what do you know about that person? 
- which app is that interaction happening through?
- Are they using only your APIs, or others; are they happy with your API or not? . . .
Knowing the mobile number and knowing the name, address, payment history of the person using that number, and so on, are two very different things.
 
Clearly visibility into APIs is the best aperture into your enterprise. You get valuable operations information, and even business impact information like the number of products purchased. 
 
But wouldn't it be interesting to know in what context that product was purchased; in which context the app was built, and so on? Ideally you can collect enough contextual information around the APIs so that your analytics is no longer just looking at bits and bytes of APIs. 
 
In summary - API data is great for operational and business visibility. Having contextual data greatly enriches the information for business visibility.
 
Does this combination of API data and context data lead us to a need for a BIG DATA solution for companies?
 
Yes. But let's talk about some examples of what that means. 
 
Let's just think about how your APIs are designed. Are they optimized first for back-end efficiency or for easy consumption?
They should be and probably are optimized for consumption with the understanding that the back-end optimization can come later. Optimizing for back-end systems first can lead you down a path where you build a system that's efficient from a back-end perspective, but not necessarily one that is attractive for developers to consume and adopt. 
 
This necessity to optimize for consumption tends to lead to API traffic that is chatty and voluminous. Even a small sized enterprise that has 1000 TPS can easily end up with 200 terabytes of data every year. That's not huge, but it's also very small. So just thinking about the volume and chattiness of traffic, you are definitely driving towards a big data solution. 
 
Secondly, it's also not so much about the data volume but how you interact with data.
 
Traditional data warehouses have been built to answer questions you know you have and repeatedly.
 
The new big data solutions are about determining the questions to ask. In this scenario, it's not just the volume that is at issue - but the variety of questions you need to ask. In an API, app, and customer-centric world you don't really know the patterns and questions at the outset.
 
In summary, given volume and chattiness of API traffic, and the new ways in which you need to interact with that data, I believe that we are driving to a bid data solution. 
 

Next time, we'll talk about gaining insights from that big data and the analytics on that data

In this post, I'll write out some of concepts and ideas we explored on the Webinar.

Q: What do we mean by visibility at the edge?

 in the past, enterprises could focus on controlling their interactions with customers through physical presence or websites. Now people are interacting more and more with core businesses through apps. 

Although the interaction has moved farther away and businesses are no longer in control of the interaction in the same way, customers are more important to the business than ever before. More data is coming in from developers and end users through apps and APIs.

Q: Clearly, to get such visibility, we will collect lots of data.

Before we get into the data part, are there examples of the value such visibility might provide?

Think about the number of different constituents around API interactions (even non-API interactions). There are the people transacting with your business, developers building apps, teams in enterprises responsible for the building and the success of APIs, teams responsible for the operations of APIs, and business users interested in the top and bottom lines. 

In addition to various constituents, there are lots of interactions through multiple channels with different systems, including billing, catalog, procurement, order management, and so on.

The biggest challenge is how to collect the disparate perspectives you get through these channels and generate a 360 degree sense of it. If you believe in the apps economy, you believe this will all get standardized around the APIs "interface."

Bottom line, your APIs and the management of APIs gives you visibility into a larger set of interactions than any one back-end system can.

If you harness that visibility, regardless whether you run operations for the back-end systems, your business is to make the APIs successful, or you are a business person focused on the top and bottom line, you can get a lot of insights into your business. For example: 

- How well is the API being adopted?

- What is the response time per request?

- Does operations need to allocate more resources at peak times?

- What impact is an API having on my bottom line?

and much more . . .

But there's something even more transformative one can get from API data. Instead of thinking only about what the enterprise or the API team learns from API data, think about what developers and the app end users can learn from the API data.

For example, if you are a developer, you'd like to know what impact you're having on the business whose API you are using to build your app. 

Visibility into the APIs gives insights for all of the constituents of the value chain, therefore getting us closer to a 360 degree and actionable view.

For more on the business implications, check out this recent post Engage customers where they are - at the edge of your enterprise.

Next time, we'll talk about what else is needed to make this 360 degree visibility a reality.

 

API Management Decision-Making Kit

Next Steps

 
 

Resources Gallery

News