API Security

APIs: The New Security Layer

Webcast replay

APIs provide both an extraordinary opportunity for building engaging customer experiences and for strengthening your relationship with key business partners. But they also provide potential openings for savvy hackers to get unauthorized access to customer data and perhaps even to compromise your key business systems.

In this webcast replay, Apigee chief architect Greg Brail discusses:

  • API security fundamentals
  • how to proactively watch for trouble
  • protection and mitigation strategies to keep your customers and your business safe



Security for APIs, or APIs for Security?

How many times have you driven away from your house wondering if you remembered to lock the door? Personally, I have turned my car around to check more than once, and my neighbors have gotten calls asking to check at least twice.

Our home is our prized asset. We need a door to allow friends and family to come in and out, but we also want to make sure that unwanted guests can’t enter. So we put a lock on the door — yet still we worry.

APIs as intelligent doors

APIs are like the doors to your enterprise assets. The purpose of the digital transformation most of today’s enterprises are undertaking is to have new use cases built around their most differentiated assets: their physical stores, content, and data.

Putting these assets on total lockdown does your enterprise no good. If your assets are in Fort Knox, what customer would actually go through the trouble of using them? What you want instead are intelligent doors (APIs) that open up the right assets for the right people, whether it’s the developers inside your company, trusted partners, or third developers building on your platform.

Because APIs are such a critical part of any digital strategy—and because a lack of API security would bring the digital revolution to a grinding halt—everyone using APIs puts a lot of emphasis on securing them. But how do you actually go about securing an API?

APIs as contracts

For this part of the discussion, it’s helpful to think about an API as a contract for accessing a particular door. Because an API is a contract, it is possible for the organization that offers the API to completely document and understand the interaction between the application that uses the API and the API itself. This contract-driven interaction model makes it possible for the organization that provides an API to add policies and security controls at every interaction.

An API team can therefore regulate which applications and end users are authorized to use an API and which parts of the API they are allowed to use. The team can also control what an authorized user can do, including limits on the number of API calls that can be made, or when they can be made. Finally, the team can follow the trail of API calls to understand exactly what authorized API users did, and what unauthorized attempts may have been made.

As a result, APIs, rather than presenting a new security risk, provide a well-documented and popular way for organizations to share access to data and services internally and with third parties, while also maintaining strict security controls.

APIs for programmatic access

Compared with the other ways enterprise data is shared—via a web site, file transfer, email, or even a printing press—a well-implemented API offers a far stronger set of security controls.

APIs are different because they are designed from the ground up to do only one thing, and that is to provide programmatic access to developers who code applications. Well-implemented APIs ensure that only authorized end users and applications can: access your enterprise assets; control the amount of API traffic that can be generated; ensure that API traffic does not contain malicious content; and even audit all traffic for later analysis and risk mitigation.

In other words, by creating intelligent doors, and by putting the right locks on them—and, over time, shutting off other, less secure existing windows and doggie doors—your enterprise assets become more secure.

If you are an enterprise going digital and you are concerned about the security of your assets, it’s time to consider an API-first approach to digital transformation.

Image:Arthur Shlain/The Noun Project

Security in the Digital Age: Deep-Dive Webcast

What are the biggest cyber threats facing financial and healthcare entities today and in the near future? How can organizations embrace innovation and agile development culture while balancing the time to market goals with risk management?

Join Jason Kobus, director of API banking at Silicon Valley Bank, and Apigee's head of security Subra Kumaraswamy in this webcast replay as they discuss how an effective API program, combined with a secure API management platform, can:

  • provide visibility for all security threats targeting their backend services

  • control access to sensitive data, end-to-end

  • enable developers to build secure apps with secure APIs

  • facilitate secure access with partners and developers


Industrial-Grade Data Security for Agile Development

Data breaches are making headlines on a seemingly weekly basis. With hackers exposing and exploiting millions of consumer records, data security has taken center stage among security leaders and enterprise architects. It’s a crucial time for organizations to reassess their data security architectures and risk profiles.  

The goal of every security and compliance manager should be to implement a secure-by-design pattern where all sensitive data—both in transit and at rest—are secured.

A primary reason data breaches occur: organizations lack strong data protection measures such as encryption, hashing, and tokenization, and they lack strong access control models that limit sensitive data access to authorized users. Often, they also don’t secure the cryptographic keys that are employed for data protection.

It happens over and over again: hackers get their hands on sensitive data by accessing cryptographic keys that aren't safeguarded with hardware-based encryption systems such as hardware security modules, or HSMs.

HSMs: more secure than software encryption

HSMs are physical network devices, sometimes in the form of a plug-in card that attaches directly to a server. They’re a cornerstone of digital security roots of trust, and ensure that keys are protected inside a hardened, tamper-resistant device. HSMs are considered more secure than software-based encryption systems, where keys are stored in the software.

Unlike software-based cryptographic systems, an HSM-based system is not vulnerable to attacks that target operating systems and applications that store and manage cryptographic keys within the software elements.

Balancing security with self-service

Organizations handling data security often grapple with trying to implement strong security control without creating too much complexity.

So how can developers continue to be agile and innovate their next mobile or IoT app while protecting sensitive data? Developers prefer self-service and easy-to-use APIs to help build engaging apps and experiences. But the majority of the developers aren’t trained in secure design and coding practices and don't possess the skills to develop the industrial-grade cryptographic systems required to support an organization’s data protection needs.

Successful IT organizations understand that good security can be achieved when developers consume security-as-a-service in the form of APIs. This makes it essential for organizations to choose an API management platform that is not only capable of supporting strong security controls such as HSM, but also can abstract and hide complexity from the developers who consume APIs that handle sensitive data.

Where are your keys?

Developers interacting with Apigee Edge, for example, can store and manage the cryptographic keys in a secure way using a commercial HSM module. What are the typical keys used by the platform when protecting sensitive data that crosses the trust boundary?

  • Keys that authenticate and encrypt two-way SSL/TLS sessions
  • Keys that encrypt XML/JSON payloads when dispatched to a destination service
  • Keys used for signing REST payloads

In all the these cases, cryptographic keys should be stored in HSMs offered by the likes of Safenet, Thales, or Amazon’s AWS CloudHSM, and should be governed by strong access control.  

Improve adoption, embed security, quell breaches

Any enterprise data protection strategy and architecture must consider strong cryptographic systems to help protect sensitive data. Data protection using HSM is a recommended approach for organizations dealing with customers' personally identifiable information (PII) or sensitive or regulatory data—such as financial institutions, healthcare organizations, and government agencies.

And when the protection mechanisms and configuration are made available to developers via APIs, it improves developer adoption of security and automatically embeds security into applications. While these measures alone don’t guarantee absolute data security, they can make security breaches far less likely.

For more on API security, check out the webcast replay, "How to Achieve Agile API Security."

Photo: "T"eresa/Flickr

The IRS Breach and the Importance of Adaptive API Security

The recent Internal Revenue Service data breach, which used the “Get Transcript” API to access the agency’s data, highlights the increased sophistication of cyber attacks that aim to profit from taxpayers’ personal information. The API in question was used by the IRS’ browser-based app to remotely access IRS systems and was implemented to deliver line-by-line tax return information, account details (marital status and income adjustments, for example), and wage and income statements.

In the United States, tax data is very valuable to cyber criminals, as it can be abused to receive phony refunds (this kind of fraud costs the U.S. approximately $6 billion annually) and commit mortgage fraud, identity theft, and other crimes. The IRS wasn’t able to detect and block this latest attack because its back-end application was authorizing users using personally identifiable information (PII) such as social security numbers, which can be gathered through other channels, phishing attacks, or from black markets that sell PII data. So the agency’s intrusion detection system wasn’t able to distinguish a legitimate taxpayer access request from a fraudulent request.

The trouble with knowledge-based authentication

The IRS could have employed stronger authentication schemes, such as two-factor authentication using smartphones, but this entails higher implementation and support costs, and can potentially impede user adoption due to a degraded user experience and a lack of phone capabilities such as SMS. Instead, the IRS used a simple authentication mechanism that has a very low barrier to adoption. Unfortunately, it also made it much easier for bots and cyber criminals to steal taxpayer data, and it did so with a 50% success rate.

So how do you verify that the user who claims to be John Doe is indeed John Doe, when your mode of authentication is a simple username and password combination? Is combining this with an additional layer of authorization based on user attributes such as knowledge based user attributes enough to keep cyber thieves at bay?

The short answer is “no.” This type of knowledge-based authentication is becoming less relevant in today’s world, where the knowledge is no longer a secret. It has become pretty easy for cyber criminals to dig up relevant PII and assume a taxpayer’s identity.

So how can organizations protect their crown jewels while making it easy for users to access their data or transact in a secure way? How can you instill confidence and trust in consumers who are eager to access government, commerce, financial, and healthcare services using smart phones and personal computers?

The adaptive security approach

Organizations have to adopt an adaptive security approach—one that learns consumer behavior and automatically detects and blocks any cyber criminal or bots that may have gained access to consumer usernames, passwords, and other personal attributes used for knowledge-based authentication.

Using machine learning and statistical models, an adaptive security system constantly learns “good behaviors,” which helps it distinguish”bad behaviors” and enforce dynamic policies that block bots from accessing a protected resource (the web, an API, or a datastore).

Bad behaviors manifest in the form of anomalous activities, including:

  • Systematic walk-throughs of the application resource paths by bots

  • Requests originating from a bot network or low-reputation IP address or ISP or compromised proxies and devices (rooted Android devices or PCs with malware, for example)

  • High rates of access from certain IP addresses or end points

  • High rates of access to URIs (resources) that are infrequently accessed by end users (password or home address changes, for example)

  • High rates of form submissions with slight variations in the input parameters (bots using brute force techniques with authentication API calls using random user IDs, for example)

  • High error rates on access to resources, especially those that are available to privileged users or applications

Gaining visibility into API access anomalies by way of bot activities is an important consideration when implementing API security and fraud protection program. Bots, also commonly referred to as content scrapers or spiders, are increasingly targeting retailers, e-commerce service providers, and anyone who has valuable content.

Studies show that bot activity represents more than half of overall Internet traffic. Bots are known to target retailers with dynamic pricing, loyalty programs, financial services, services with copyright materials, and intellectual property such as visual designs. They can have a major load impact on a website and API backend infrastructure, create performance headaches for operators, and hurt a company’s brand and bottom line due to content theft or competitive information scraping.

Advanced API analytics 

At Apigee we’ve implemented an adaptive security system using the advanced API analytics capabilities of the Apigee Edge API management product and our Insights predictive analytics product. This advanced analytics functionality provides our customers visibility into bot activities or brute-force attacks against APIs using stolen keys or user account details.

For example, if an API key is breached and abused by hackers to access the APIs in an unauthorized fashion, the bot detection system will detect the anomaly and flag the end points as bots.  Furthermore, a full bot protection architecture can be implemented in Edge using an API that automatically imports blacklists from the bot analytics system. Using this closed loop mechanism, bots can be blocked or throttled by Edge, protecting the origin servers from bot traffic.

This adaptive API security system is designed to protect consumers and enterprises to thwart dynamic threat vectors that are hard to detect otherwise using static policies.

In summary, an effective API security program requires a layered security approach that combines traditional security controls (authentication, authorization, auditing, encryption, and threat protection) with an adaptive security control that continuously learns and protects APIs from malicious bots. Organizations that invest in the adaptive security approach protect their brand image, reduce fraud and content theft, and safeguard their customer data from malicious bots on the internet.

API Security Deep Dive (webcast & podcast)

Threat protection and application access controls are key security mechanisms that protect APIs when exposed to internal or external users and developers.
In this technical deep-dive webcast, Apigee's Subra Kumaraswamy and Chris Von See discuss API threats and the protection mechanisms that every API and app developer must implement for safe and secure API management. 
This webcast covers: 

  • the API threat model 
  • how to design and implement appropriate guardrails for API security using built-in policies and configuration 
  • a demo of Apigee Edge threat protection features, including TLS encryption, XML/JSON/SQL injection attacks, and rate limiting 

Whether you're an IT security architect or an API or app developer, this webcast will help you understand secure API management. 


API-First Security: Don't Build Your Own Maginot Line

The Maginot Line is remembered as one of history’s most costly strategic blunders. France’s massively complex, heavily fortified, and expensive 940-mile line was built before WWII to keep Germany at bay. But France’s foes simply skirted the fortification via Belgium, and conquered France in six weeks.

The Maginot Line failed because it wasn’t designed to counter the newer threats and tactics employed by Germans at that time; it wasn’t flexible enough to adapt to new, nimbler threats.

The Maginot Line (image: Wikimedia.org)

Digital security strategies can exhibit a similar weakness, as they’re not designed to adapt to new and unexpected threats (the Heartbleed vulnerability is a good example).

Omnichannel security

A typical enterprise has hundreds of business applications hosted in private or public clouds that interact with their users (customers, partners, and employees) spread across geographies and time zones. These interactions take place via a variety of channels: web, mobile, APIs, VPNs, cloud services, and sometimes via contactless payment terminals supporting Apple Pay.

As a consequence, enterprises need to plan for an omnichannel user engagement model that provides a consistent user experience, with security and privacy built into all channels. And it should be supported by a security architecture that’s responsive enough to enable enterprises to prevent, detect, and react to newer threats, in near-real time.

The problem with bolt-on security

One way to achieve this outcome is to secure each channel independently using point solutions. This approach usually involves a bolt-on security model, in which the protection mechanism addresses specific threats for individual channels—securing mobile apps using a custom authentication without OAuth, SSO (Single Sign-on), and support for federated identity store, for example.

With this approach, a business won’t be able to support new security requirements as user engagement moves from a single channel to multi-channel mode. And, from the perspective of the chief information security officer, a channel-specific security approach may provide the Maginot Line-like perception of strong fortifications, but won’t be able to withstand new threats that blend across channels. Ultimately, the channel-specific security strategy leads to security weakness, with inconsistent user experiences across channels.  

API-first architecture

An effective digital security strategy and architecture that adapts quickly to new threats can be realized by building on an API-first architecture. In this model, every mobile, web, and IoT device interacts with business applications via APIs. An API-first architecture facilitates interactions within and across channels via an API tier that’s distributed to scale with user engagement. By building security into the API tier, an enterprise can enforce a consistent protection mechanism across all channels.

One of the advantages of API-first architecture is the ability to expose APIs as “products” with built-in authentication, authorization, and threat protection. API products enable the right level of security for users, irrespective of the channel through which the interaction takes place.

For example, a "private" API product can expose private APIs which support SSO for employees using a corporate active directory, while a "public" API product supports SSO using a federated or social identity. As a result, you can manage security across channels in a very consistent way while preserving the agility of the business units exposing different sets of APIs.

Screen Shot 2015-04-07 at 1.02.28 PM.png

API-first security principles

Every enterprise should follow API-centric security principles, as they are a critical part of an API-first architecture:

  • Expose your APIs as API products with fine granular access control built in. Your mobile app designed for users of your public API product can support social logins, such as Facebook. A mobile app designed for your employees using a private API product may use your corporate directory for authentication and authorization needs. Both products should support OAuth and authorization scope to limit access to different sets of APIs.

  • Always configure security; code only when absolutely necessary. Using the API product approach, you can configure security policies that support authentication, authorization, and threat protection that persist across channels. Security configuration provides a better experience for developers and helps developers adapt security features faster. Threat protection features such as rate limiting and OWASP Top-10 Injection protection act as a defense layer for all channel interactions. But you’ll still need the option to code security for cases where configuration may not be an option. For example, you may need to query your internal SIEM (security information event management) system for malicious sets of IP addresses (black lists) and block the threats across all channels.

  • Log all API interactions across channels to get an end-to-end view of user, device and app activities. This approach helps provide correlations of user activities across all channels and identify any abnormal behavior that deserves investigation. API analytics can provide insights into traffic spikes from certain devices, apps, IP addresses, or geographies.

  • Always layer API security with other controls for a defense-in-depth strategy. Be sure to employ additional access protection between apps and API services. An example: using IP-based whitelisting and two-way SSL/TLS in addition to OAuth-based security.

Remember, build your apps and services upon an API-first architecture that’s resilient and adaptive to new threats. An API-first architecture enables consistent security policy enforcement across channels while improving user engagement and developer productivity.

With an API-first architecture approach, you can avoid the digital Maginot Line, where hackers exploit the weakest channel and create a costly data breach response situation for your organization.


Secure your API with a Layer of Indirection

A popular principle among computer scientists states that every problem can be solved by adding a layer of indirection. When it comes to API and cloud security, this principle can work wonders. A layer of indirection (also referred to as a layer of abstraction) in the cloud with security built into the platform can help mitigate threats including the recently discovered “FREAK” SSL/TLS vulnerability, and can ease concerns for enterprises who migrate workload or services to the cloud.

The FREAK vulnerability, which emerged last week, enables attackers to intercept HTTPS connections between vulnerable clients and servers and force them to use “export-grade” cryptography, which can then be decrypted or altered.

With the emergence of threats like this, the ability to configure security at the edge of your enterprise—quickly—has become critical. With API security, the primary goal is to protect the APIs exposed, from your data centers and cloud services, to your developers and partners. So how do you do this, without compromising your application security but allowing app developers to innovate at warp speed?

Here is where applying the principle of indirection to your APIs by using an API management platform offers protection in a seamless way. An API indirection layer with a strong security facade helps developers consume APIs in a frictionless way. It also enables API teams to govern the APIs with consistent security aligned with their enterprise security architecture. The benefits of a secure API facade on your APIs include the ability to:

  • mitigate any application security weakness by taking advantage of the API management platform's threat protection features, including 2-way TLS, OWASP Top 10 protection, rate limiting, and quota restrictions for private or public apps

  • provide a strong authentication mechanism using OAuth 2.0 for mobile and web apps, even though your API doesn’t support newer and stronger security protocols

  • support fine-grained authorization for your APIs for developers and partners without requiring you to redesign your APIs

  • buy additional time to patch your APIs from zero-day security threats such as Poodle, Ghost, and FREAK attacks; a secure facade facilitates configuration in seconds instead of the days required to code and deploy security controls necessary to protect your vulnerable APIs

So when the next zero-day security vulnerability rears its ugly head, you can simply configure your blacklist to block the vulnerable protocols (such as the Poodle vulnerability in SSL3.0), ban the weak ciphers (like the FREAK vulnerability, which is due to weak export cipher suites), and be assured of your API protection.

The strength of an API security program is not only determined by prevention controls such as OAuth-based authentication, but also your ability to react to new zero-day security threats such as FREAK. We’ll witness more of these attacks; the only way to future proof your security response is to create an architecture that enables you to configure security at the edge of your enterprise in near real-time, without impacting your enterprise’s mission-critical APIs.

Photo: Holly Victoria Norval/Flickr

Tutorial: Using Remote Services and API Key Security

In the past few Apigee-127 tutorials, we’ve explored core features like OAuth 2.0 security, quotas, and caching—key ingredients for building enterprise-quality APIs. Here, we'll explore two new Apigee-127 features: remote services and API key security. We’ll show you how to create a remote service and bind it to an Apigee-127 project. Then, we’ll use that service to handle API key security validation for the API.

What are remote services?

A remote service is available to an Apigee-127 project no matter where the project is deployed. Currently, the only Apigee-127 service available is the RemoteProxy service, which is used with the Apigee OAuth provider. It provides endpoints for creating and verifying OAuth 2.0 tokens, and it's deployed on the Apigee Edge platform.

What is API key security?

An API key enables an API provider to uniquely identify a developer app. Typically, an app developer must register an app with the API provider to obtain the key, which remains confidential between the two parties.

Whenever the app calls the API, it typically passes the key either in a query parameter or header. For example:

If the key is valid, the call proceeds; otherwise, an error is returned.

What version of Apigee-127 do I need?

To use remote services and API keys, you need to install Apigee-127 v0.11.0, or a later version. Verify your version with the command:

$ a127 -- version

Check out the doc wiki for installation instructions and release notes.

Before you begin

The currently active Apigee-127 account must have the "apigee" provider. You can list all accounts, check the current account properties, and create new accounts with the Apigee-127 command-line tool. Just enter $a127 account -h for command options.

Create a new service

There are two ways to create a new service:

  • when you create a new account, you have the option of also creating a service

  • you can use the Apigee-127 command-line tool’s service option to create services directly

Let's look at the command-line tool option:

$ a127 service create
[?] Service Type? RemoteProxy
[?] Service name? RemoteProxyBlog
Creating service RemoteProxyBlog from myaccount...
Remember to bind your service to any projects that require it.


 account: myaccount
 type: RemoteProxy
 uri: 'http://myorg-prod.apigee.net/remoteproxyblog'
 key: f34RUcMxFGwTe6e5KnpZIJfT

You’ll need to remember the service name (remoteproxyblog in this example) for later when you configure your API.

Here's what happens behind the scenes:

  • an API proxy called RemoteProxyBlog is deployed to Apigee Edge

  • a developer, a product, and a developer app are created on Apigee Edge

  • Apigee-127 retrieves the proxy URI and key from the developer app, along with some other information, and stores it in ~/.a127/services; this information enables your Apigee-127 project to communicate with the service

Bind the service to a project

Next, use this a127 command to bind the new service to your project. This binding enables the service to be used wherever the project is deployed:

$ a127 project bind RemoteProxyBlog

Service RemoteProxyBlog bound to Project myblogproject.
If this project is deployed, you will need to redeploy to activate the new configuration.

As the message says, if your project was already deployed (to Apigee Edge, for instance), you need to redeploy it. If you’re running your API project locally, you don’t have to deploy your project.

Configure an API key policy in your Swagger file

Next, we add an API key “policy” to the API by editing the project’s ./api/swagger/swagger.yaml file. No coding is required!

Let’s go step by step. First, the API key policy requires a Swagger security definition. You can add this stanza at the end of the Swagger file, after the response definitions section, like this:

       type: apiKey
       name: apiKey
       in: query

We’re specifying a name for the security policy (apiKeyQuery), the type of policy (apiKey), and how the key will be passed in API requests (a query parameter named apiKey).

Next, configure the parameters for the RemoteProxy service in the x-a127-config section of the project’s Swagger file. This is where you must supply the name of the RemoteProxy service and the required key and uri parameters, like this:

     RemoteProxyBlog.key: &apigeeProxyKey CONFIGURED
     RemoteProxyBlog.uri: &apigeeProxyUri CONFIGURED

Now declare the API key policy in the x-a127-services section of your project’s swagger.yaml file. The policy options are the key and uri references that you configured previously. The name of the policy (specified in the security definition) is apiKeyQuery, and you'll reference that name when you add the policy to an API path.

       provider: volos-oauth-apigee
         key: *apigeeProxyKey
         uri: *apigeeProxyUri

Finally, add the security policy to one or more API paths. Here's a sample path, with the apiKeyQuery policy applied to the GET operation (the last two lines):

       # binds a127 app logic to a route
       x-swagger-router-controller: hello_world
       x-a127-apply: {}
         description: Returns 'Hello' to the caller
         # used as the method name of the controller
         operationId: hello
           - apiKeyQuery: []

That's it! Now, let’s see how to call the API with an API key. You can find a working example of this code here: https://github.com/apigee-127/a127-samples/tree/master/security-apikey-apigee.

Get the API key

Where does the API key come from? It’s a valid client ID (also called a consumer key) associated with a developer app registered on Apigee Edge. On Apigee Edge, you can find developer apps in the UI under Publish > Developer Apps.

The way this typically works in production is this: a developer first registers an app with Apigee Edge. This can be done through a developer portal, or through the Edge management interface. In return, the developer receives the client ID. That value is the API key.

Any valid client ID within the organization specified in your Apigee-127 account will work (for example, prod or test). For testing purposes, you can use the key that was returned when you created the RemoteProxy service, because that key represents a valid client ID.

Call the API

To call the API, just add an apiKey query parameter, passing it a valid API key, like this:

$ curl


Hello, Will

For more information

Check out the a127 doc wiki for more details on API key, basic auth, and OAuth 2.0 security options. You can also find a working sample on GitHub here: https://github.com/apigee-127/a127-samples/security-apikey-apigee.


Lessons from a Customer Security Incident

Building products used by demanding customers is hard. Running operations is hard. Safeguarding customer data and infrastructure is hard. We’ve benefitted from all who have traveled on this journey before us, and we’d like to share our experiences and lessons learned: what works, and what doesn’t.

In this post, we share a summary of a recent security incident and the lessons we learned from handling the attack.

In early December, an unauthorized party accessed credentials for two of our customers for accessing the Apigee Management Service. These credentials were stored in a third-party hosting service, which hosts our developer portal. The intruder accessed the credentials by exploiting a known Drupal vulnerability before the fix patch was applied, and then made a few configuration changes using the compromised credentials. Since these changes were unusual, they caught our attention. Following our standard first-response security procedures, we acted quickly to protect our customers by disabling public access to our Management Service.  

We then performed a thorough investigation to assess the extent of the attack and found it was limited to the two customers and the set of configuration changes made. We restored all the customers’ original configuration from backups and took steps to prevent similar attacks.

A painful lesson learned from this incident: we weren’t following the principle of "least privilege.” The credentials stored in the third-party service had more privilege than they should have. Had we been more diligent about this, the intruder would have had a much harder time accessing the credentials.

The incident raises several important security best practices to keep in mind:

  • Limit your user list to those who actually need to use the product

  • Periodically review your user access list to ensure an up-to-date user list; delete users no longer associated with your organization or the job function

  • Use RBAC (role based access control) to assign users the level of access they need to get their jobs done and no more—we’ve defined several roles to make it easy for you to do this

In our cloud offering, we run tens of billions of API calls per month that interact with our customers’ back-end systems. Our on-premises software handles an equally large number. The lessons we’ve learned and the myriad architectural choices we’ve made enable our customers to: scale their API traffic (both stateless and stateful); create isolation in a multi-tenant environment; provide flexibility to configure or code; support an analytical system that doesn't require tuning as apps and APIs change; and enforce security with access control and threat mitigation.

For more on security best practices, see the following articles and posts:

Apigee is hiring. If you’re an engineer excited about tackling challenging problems, visit our careers page.