Load balance API traffic across multiple backend servers

Was this helpful?

One way that Apigee enhances the availability of your API is by providing built-in support for load balancing and failover across multiple backend server instances. 

Back to top

Step 1: Create target servers in an environment

To configure load balancing and failover, you create two or more named TargetServer configurations that you then reference in the HTTPConnection element in a TargetEndpoint configuration. 

A TargetSever definition consists of a name, a host and a port, with an additional element to indicate whether the target server is enabled or disabled. (When defining the Host element, omit the protocol prefix http:// or https://. The protocol is set dynamically based on whether outbound SSL is configured for the TargetEndpoint.)

Sample Target Server Configuration:

<TargetServer name="target1">
  <Host>1.mybackendservice.com</Host>
  <Port>80</Port>
  <IsEnabled>true</IsEnabled>
</TargetServer>

To create a target server in an environment:

$ curl -H "Content-Type:text/xml" -X POST -d \
'<targetserver name="target2">
  <host>2.mybackendservice.com</host>
  <port>80</port>
  <isenabled>true</isenabled>
 </targetserver>' \
-u myname:mypass https://api.enterprise.apigee.com/v1/o/{myorg}/environments/test/targetservers

Sample Response:

{
  "host" : "1.mybackendservice.com",
  "isEnabled" : true,
  "name" : "target1",
  "port" : 80
}

Create a second target server:

$ curl -H "Content-type:text/xml" -X POST -d \
'<targetserver name="target2">
  <host>2.mybackendservice.com</host>
  <port>80</port>
  <isenabled>true</isenabled>
</targetserver>' \
-u myname:mypass https://api.enterprise.apigee.com/v1/o/{myorg}/environments/test/targetservers

Sample Response:

{
  "host" : "2.mybackendservice.com",
  "isEnabled" : true,
  "name" : "target2",
  "port" : 80
}

Retrieve a list of target servers in an environment:

$ curl -u myname:mypass https://api.enterprise.apigee.com/v1/o/{myorg}/environments/test/targetservers

Sample response:

[ "target2", "target1" ]

There are now two target servers available for use by APi proxies deployed in the test environment. To load balance traffic across these target servers, you configure the HTTP connection an API proxy's target endpoint to use the target servers.

There is no limit on the number of target servers that you can set up in an environment or reference from a target endpoint.

Back to top

Step 2: Configure target endpoint to load balance across named target servers

Now that you have two target servers available, you can modify the target endpoint HTTP connection setting to reference those two target servers by name.

<TargetEndpoint name="default">
    <HttpTargetConnection>
        <LoadBalancer>
            <Server name="target1" />
            <Server name="target2" />
        </LoadBalancer>
    </HttpTargetConnection>
</TargetEndpoint>

The configuration above is the most basic load balancing configuration possible. The load balancer supports three load balcing algorithms, Round Robin, Weighted, and Least Connection. Round Robin is the default algorithm. Since no algorithm is specified in the configuration above, outbound requests from the API proxy to the backend servers will alternate, one for one, between target1 and target 2. 

Back to top

Step 3: Set load balancer options

You can tune availability by using options for load balancing and failover at the load balancer and target server level.

Available options for load balancers are:

 

Algorithm

Sets the algorithm used by load balancer

Round Robin

The default algorithm, forwards request to each target server in the order in which the servers are listed in the target endpoint HTTP connection.

For example:

<TargetEndpoint name="default">
  <HttpTargetConnection>
      <LoadBalancer>
        <Algorithm>RoundRobin</Algorithm>
        <Server name="target1" />
        <Server name="target2" />
      </LoadBalancer>
  </HttpTargetConnection>
</TargetEndpoint>

 

Weighted

The weighted load balancing algorithm enables you to configure proportional traffic loads for your target servers. The weighted load balancer distributes request to your target servers in direct proportion to each target server's weight. Therefore, the weighted algorithm requires you to set a<weight> attribute for each target server. For example:

<TargetEndpoint name="default">
  <HttpTargetConnection>
      <LoadBalancer>
        <Algorithm>Weighted</Algorithm>
        <Server name="target1">
          <Weight>1</Weight>
        </Server>
        <Server name="target2">
          <Weight>2</Weight>
        </Server>
      </LoadBalancer>
  </HttpTargetConnection>
</TargetEndpoint>

In this example, 2 requests will be routed to target2 for every 1 request routed to target1.

Least Connection

Load balancers configured to use the least connection algorithm route outbound requests to the target server with fewest open HTTP connections.

<TargetEndpoint name="default">
  <HttpTargetConnection>
      <LoadBalancer>
        <Algorithm>LeastConnection</Algorithm>
        <Server name="target1" />
        <Server name="target2" />
      </LoadBalancer>
  </HttpTargetConnection>
</TargetEndpoint>

Maximum Faliures

The maximum number of failed requests from the API proxy to the target server that results in the request being redirected to another target server. As configured below, target1 will be removed from rotation after 5 failed requests.

<TargetEndpoint name="default">
  <HttpTargetConnection>
      <LoadBalancer>
        <Algorithm>RoundRobin</Algorithm>
        <Server name="target1" />
           <MaxFailures>5</MaxFailures>
        <Server name="target2" />
      </LoadBalancer>
  </HttpTargetConnection>
</TargetEndpoint>

Retry

A boolean, retry defines whether whether a target server is retired after a failure. By default, this is set to true. If set to false, a failure by the target server results in a failure for the transaction, with a failure result returned to the requesting client app.

<RetryEnabled>true</RetryEnabled>

Fallback

One (and only one) target server can be set as the 'fallback' server. The fallback target server is not included in load balancing routines until all other target servers are identified as unavailable by the load balancer. When the load balancer determines that all target serversare unavailable, all traffic is routed to the fallback server. For example:

<TargetEndpoint name="default">
  <HttpTargetConnection>
      <LoadBalancer>
        <Algorithm>RoundRobin</Algorithm>
        <Server name="target1" />
        <Server name="target2" />
        <Server name="target3"
          <IsFallback>true</IsFallback>
      </LoadBalancer>
  </HttpTargetConnection>
</TargetEndpoint>

The configuration above results in round robin load balancing between targets 1 and 2 until both targets 1 and 2 are unavailable. When targets 1 and 2 are unavailable, all traffic is routed to target 3.

IsEnabled

A boolean, enables target servers to be turned on and off by changing this setting from true to false. A common usage would be to write an app or script that enables or disables target servers automatically based on expected capacity requirements, maintenance schedules, etc.

<IsEnabled>true</IsEnabled>
Back to top

Next Steps

If you have questions, post to the Apigee Developer Forum.

Back to top