When a client reaches its API usage limits, API rejects the request by returning theHTTP 429 Too Many Requests error to the client. The client may retry after the retry period that is usually returned in a custom HTTP response header. This is an API throttling strategy commonly employed.
There are situations where API may depend on an external service provider that may have a fixed capacity. As this external dependency has a fixed capacity and cannot handle bursts in requests, we have to control the throughput of requests to meet the service level agreements of the dependency. In this article, we will explore two alternate strategies to throttle API usage to deal with this condition: