Overview of Retry Policies in the Windows Azure Storage Client Library
2012-01-11 14:35
513 查看
http://blogs.msdn.com/b/windowsazurestorage/archive/2011/02/03/overview-of-retry-policies-in-the-windows-azure-storage-client-library.aspx
The RetryPolicies in the Storage Client Library are used to allow the user to customize the retry behavior when and exception occurs. There are a few key points when using RetryPolicies that users should take into consideration, the first is when they are evaluated,
and the second is what the ideal behavior for your scenario is.
When the Storage Client Library processes an operation which results in an exception, this exception is classified internally as either “retryable” or “non-retryable”.
“Non-retryable” exceptions are all 400 ( >=400 and <500) class exceptions (Bad gateway, Not Found, etc.) as well as 501 and 505.
All other exceptions are “retryable”. This includes client side timeouts.
Once an operation is deemed retryable the Storage Client Library evaluates the RetryPolicy to see if the operation should be retried, and if so what amount of time it should backoff (sleep) before executing the next attempt. One thing to note is that if an
operation fails the first two times and succeeds on the third the client will not see the exception as all previous exceptions will have been caught. If the operation results in an error on its last attempt is an exception then the last caught exception is
rethrown to the client.
Also, please note that the timeout that is specified is applied to each attempt of a transaction; as such an operation with a timeout of 90 seconds can actually take 90 * (N+1) times longer
where N is the number of retry attempts following the initial attempt.
There are three default RetryPolicies that ship with the Storage Client Library listed below. See http://msdn.microsoft.com/en-us/library/microsoft.windowsazure.storageclient.retrypolicies_members.aspx for
full documentation
RetryPolicies.NoRetry –
No retry is used
RetryPolicies.Retry –
Retries N number of times with the same backoff between each attempt.
RetryPolicies.RetryExponential (Default)
– Retries N number of times with an exponentially increasing backoff
between each attempt. Backoffs are randomized with +/- 20% delta to avoid numerous clients all retrying simultaneously. Additionally each backoff is between 3 and 90 seconds per attempt (RetryPolicies.DefaultMinBackoff,
and RetryPolicies.DefaultMaxBackoff respectively) as such an operation can take longer than RetryPolicies.DefaultMaxBackoff. For example let’s say you are on a slow edge connection and you keep hitting a timeout error. The first retry will occur after ~ 3sec
following the first failed attempt. The second will occur ~ 30 seconds following the first retry, and the third will occur roughly 90 seconds after that.
In addition to using the standard retry polices detailed above you can construct a custom retry policy to fit your specific scenario. A good example of this is if you want to specify
specific exceptions or results to retry
for or to provide an alternate backoff algorithm.
The RetryPolicy is actually a delegate
that when evaluated returns a Microsoft.WindowsAzure.StorageClient.ShouldRetry delegate. This syntax may be a bit unfamiliar for some users, however it provides a lightweight mechanism to construct state-full retry instances in controlled manner. When
each operation begins it will evaluate the RetryPolicy which will cause the CLR to create a state object behind the scenes containing the parameters used to configure the policy.
Example 1: Simply linear retry policy
The Highlighted blue code conforms to the
Microsoft.WindowsAzure.StorageClient.RetryPolicy delegate type; that is a function that accepts no parameters and returns a Microsoft.WindowsAzure.StorageClient.ShouldRetry delegate.
The highlighted yellow code conforms to the signature for the Microsoft.WindowsAzure.StorageClient.ShouldRetry delegate and will contain the specifics of your implementation.
Once you have constructed a retry policy as above you can configure your client to use it via
Cloud[Table/Blob/Queue].Client.RetryPolicy = LinearRetry(<retryCount, intervalBetweenRetries>).
Example 2: Complex retry policy which examines the last exception and does not retry on 502 errors
Note the additional argument statusCodesToFail, which illustrates the point that you can pass in whatever additional data to the retry policy that you may require.
Example 3: A custom Exponential backoff retry policy
In example 3 above we see code similar to the default exponential retry policy that is used by default by the Windows Azure Storage Client Library. Note the parameters minBackoff and maxBackoff. Essentially the policy will calculate a desired backoff and then
enforce the min / max boundaries on it. For example, the default minimum and maximum backoffs are 3 and 90 seconds respectively that means regardless of the deltaBackoff or increase the policy will only yield a backoff time between 2 and 90 seconds.
We strongly recommend using the exponential backoff retry policy provided by default whenever possible in order to gracefully backoff the load to your account, especially if throttling was to occur due to going over the scalability
targets posted here. You can set this manually by via [Client].RetryPolicy = RetryPolicies.RetryExponential(RetryPolicies.DefaultClientRetryCount, RetryPolicies.DefaultClientBackoff).
Generally speaking a high throughput application that will be making simultaneous requests and can absorb infrequent delays without adversely impacting user experience are recommended to use the exponential backoff strategy detailed above. However for user
facing scenarios such as websites and UI you may wish to use a linear backoff in order to maintain a responsive user experience.
The RetryPolicies in the Storage Client Library are used to allow the user to customize the retry behavior when and exception occurs. There are a few key points when using RetryPolicies that users should take into consideration, the first is when they are evaluated,
and the second is what the ideal behavior for your scenario is.
When the Storage Client Library processes an operation which results in an exception, this exception is classified internally as either “retryable” or “non-retryable”.
“Non-retryable” exceptions are all 400 ( >=400 and <500) class exceptions (Bad gateway, Not Found, etc.) as well as 501 and 505.
All other exceptions are “retryable”. This includes client side timeouts.
Once an operation is deemed retryable the Storage Client Library evaluates the RetryPolicy to see if the operation should be retried, and if so what amount of time it should backoff (sleep) before executing the next attempt. One thing to note is that if an
operation fails the first two times and succeeds on the third the client will not see the exception as all previous exceptions will have been caught. If the operation results in an error on its last attempt is an exception then the last caught exception is
rethrown to the client.
Also, please note that the timeout that is specified is applied to each attempt of a transaction; as such an operation with a timeout of 90 seconds can actually take 90 * (N+1) times longer
where N is the number of retry attempts following the initial attempt.
Standard Retry Policies
There are three default RetryPolicies that ship with the Storage Client Library listed below. See http://msdn.microsoft.com/en-us/library/microsoft.windowsazure.storageclient.retrypolicies_members.aspx forfull documentation
RetryPolicies.NoRetry –
No retry is used
RetryPolicies.Retry –
Retries N number of times with the same backoff between each attempt.
RetryPolicies.RetryExponential (Default)
– Retries N number of times with an exponentially increasing backoff
between each attempt. Backoffs are randomized with +/- 20% delta to avoid numerous clients all retrying simultaneously. Additionally each backoff is between 3 and 90 seconds per attempt (RetryPolicies.DefaultMinBackoff,
and RetryPolicies.DefaultMaxBackoff respectively) as such an operation can take longer than RetryPolicies.DefaultMaxBackoff. For example let’s say you are on a slow edge connection and you keep hitting a timeout error. The first retry will occur after ~ 3sec
following the first failed attempt. The second will occur ~ 30 seconds following the first retry, and the third will occur roughly 90 seconds after that.
Creating a custom retry policy
In addition to using the standard retry polices detailed above you can construct a custom retry policy to fit your specific scenario. A good example of this is if you want to specifyspecific exceptions or results to retry
for or to provide an alternate backoff algorithm.
The RetryPolicy is actually a delegate
that when evaluated returns a Microsoft.WindowsAzure.StorageClient.ShouldRetry delegate. This syntax may be a bit unfamiliar for some users, however it provides a lightweight mechanism to construct state-full retry instances in controlled manner. When
each operation begins it will evaluate the RetryPolicy which will cause the CLR to create a state object behind the scenes containing the parameters used to configure the policy.
Example 1: Simply linear retry policy
public static RetryPolicy LinearRetry(int retryCount, TimeSpan intervalBetweenRetries) { return () => { return (int currentRetryCount, Exception lastException, out TimeSpan retryInterval) => { // Do custom work here // Set backoff retryInterval = intervalBetweenRetries; // Decide if we should retry, return bool return currentRetryCount < retryCount; }; }; }
The Highlighted blue code conforms to the
Microsoft.WindowsAzure.StorageClient.RetryPolicy delegate type; that is a function that accepts no parameters and returns a Microsoft.WindowsAzure.StorageClient.ShouldRetry delegate.
The highlighted yellow code conforms to the signature for the Microsoft.WindowsAzure.StorageClient.ShouldRetry delegate and will contain the specifics of your implementation.
Once you have constructed a retry policy as above you can configure your client to use it via
Cloud[Table/Blob/Queue].Client.RetryPolicy = LinearRetry(<retryCount, intervalBetweenRetries>).
Example 2: Complex retry policy which examines the last exception and does not retry on 502 errors
public static RetryPolicy CustomRetryPolicy(int retryCount, TimeSpan intervalBetweenRetries, List<HttpStatusCode> statusCodesToFail) { return () => { return (int currentRetryCount, Exception lastException, out TimeSpan retryInterval) => { retryInterval = intervalBetweenRetries; if (currentRetryCount >= retryCount) { // Retries exhausted, return false return false; } WebException we = lastException as WebException; if (we != null) { HttpWebResponse response = we.Response as HttpWebResponse; if (response == null && statusCodesToFail.Contains(response.StatusCode)) { // Found a status code to fail, return false return false; } } return currentRetryCount < retryCount; }; }; }
Note the additional argument statusCodesToFail, which illustrates the point that you can pass in whatever additional data to the retry policy that you may require.
Example 3: A custom Exponential backoff retry policy
public static RetryPolicy RetryExponential(int retryCount, TimeSpan minBackoff, TimeSpan maxBackoff, TimeSpan deltaBackoff) { // Do any argument Pre-validation here, i.e. enforce max retry count etc. return () => { return (int currentRetryCount, Exception lastException, out TimeSpan retryInterval) => { if (currentRetryCount < retryCount) { Random r = new Random(); // Calculate Exponential backoff with +/- 20% tolerance int increment = (int)((Math.Pow(2, currentRetryCount) - 1) * r.Next((int)(deltaBackoff.TotalMilliseconds * 0.8), (int)(deltaBackoff.TotalMilliseconds * 1.2))); // Enforce backoff boundaries int timeToSleepMsec = (int)Math.Min(minBackoff.TotalMilliseconds + increment, maxBackoff.TotalMilliseconds); retryInterval = TimeSpan.FromMilliseconds(timeToSleepMsec); return true; } retryInterval = TimeSpan.Zero; return false; }; }; }
In example 3 above we see code similar to the default exponential retry policy that is used by default by the Windows Azure Storage Client Library. Note the parameters minBackoff and maxBackoff. Essentially the policy will calculate a desired backoff and then
enforce the min / max boundaries on it. For example, the default minimum and maximum backoffs are 3 and 90 seconds respectively that means regardless of the deltaBackoff or increase the policy will only yield a backoff time between 2 and 90 seconds.
Summary
We strongly recommend using the exponential backoff retry policy provided by default whenever possible in order to gracefully backoff the load to your account, especially if throttling was to occur due to going over the scalabilitytargets posted here. You can set this manually by via [Client].RetryPolicy = RetryPolicies.RetryExponential(RetryPolicies.DefaultClientRetryCount, RetryPolicies.DefaultClientBackoff).
CloudTableClient tableClient = account.CreateCloudTableClient(); CloudBlobClient blobClient= account.CreateCloudBlobClient(); CloudQueueClient queueClient=account.CreateCloudQueueClient(); tableClient.RetryPolicy = RetryPolicies.RetryExponential(RetryPolicies.DefaultClientRetryCount, RetryPolicies.DefaultClientBackoff); blobClient.RetryPolicy = RetryPolicies.RetryExponential(RetryPolicies.DefaultClientRetryCount, RetryPolicies.DefaultClientBackoff); queueClient.RetryPolicy = RetryPolicies.RetryExponential(RetryPolicies.DefaultClientRetryCount, RetryPolicies.DefaultClientBackoff);
Generally speaking a high throughput application that will be making simultaneous requests and can absorb infrequent delays without adversely impacting user experience are recommended to use the exponential backoff strategy detailed above. However for user
facing scenarios such as websites and UI you may wish to use a linear backoff in order to maintain a responsive user experience.
相关文章推荐
- Windows Azure Storage Client Library 2.0 入门
- [Windows Azure] How to use the Windows Azure Blob Storage Service in .NET
- Overview of Managing Deployments in Windows Azure(staging and production environment in cloud services)
- Using ELMAH in Windows Azure with Table Storage
- Downloading the Google Cloud Storage Client Library
- How to use the Windows Azure Mobile Services (WAMS). With WAMS, signing in via Microsoft, Facebook, Twitter, and Google
- Designing Data Storage Architecture-Using the Windows Azure Storage Services
- Skipping Windows Azure Startup Tasks When Running in the Emulator
- Azure Storage Client Library 重试策略建议
- Great!The Atlas client library has been compressed in release mode.
- Azure Storage Client Library 重试策略建议
- Azure Storage Client Library 重试策略建议
- Using the Intel® MPI Library in a server/client setup
- WindowsAzure.StorageClient API
- Designing Data Storage Architecture - Introducing the Windows Azure Content Delivery Network
- Azure Storage Client Library 重试策略建议
- The difference between Web role and Worker role in windows azure platform
- Why the same files are copied to /Approot and /Siteroot in Windows Azure WebRole?
- [MSDN] Using the Windows Azure Storage Services
- windows下no swt-gtk-3550 or swt-gtk in swt.library.path, java.library.path or the jar的解决办法