Monday, March 9, 2015

Wrestling the Azure Storage REST API - Part 2

This post is about authorization HTTP header, used when making requests to Azure Storage API. There is some dependencies the previous part of this series, specially regarding the x-ms-date header field.

Authorization

The authorization field is expressed in this way:

Authorization="[SharedKey|SharedKeyLite] <AccountName>:<Signature>"

The authorization supports 2 schemes for calculating signatures, Shared Key or Shared Key Lite. The scheme you are using for authorization must be stated with either SharedKey or SharedKeyLite as the first thing in the header field.

The difference between the schemes are, Shared Key Lite is backward compatible with earlier versions of the Azure Storage API. I can't remember to have seen any example with Shared Key, I guess it is because it requires more effort to make it work.

Important!!! When using dates in authorization, these dates must be the same as x-ms-date, or else the authorisation will fail.

Shared key

Blob, Queue and File Storage signature is calculated one way, while Table Storage signature is calculated in another way. 

Blob, Queue and File Storage:

StringToSign = 
VERB + "\n" +
Content-Encoding + "\n" +
Content-Language + "\n" +
Content-Length + "\n" +
Content-MD5 + "\n" +
Content-Type + "\n" +
Date + "\n" +
If-Modified-Since + "\n" +
If-Match + "\n" +
If-None-Match + "\n" +
If-Unmodified-Since + "\n" +
Range + "\n" +
CanonicalizedHeaders +
CanonicalizedResource;

Table Storage:

StringToSign = 
VERB + "\n" +
Content-MD5 + "\n" +
Content-Type + "\n" +
Date + "\n" + 
CanonicalizedResource;

Shared key Lite

Like Shared Keys, there is a difference in calculating the keys depending on what kind of storage is used:

Blob, Queue and File Storage:

StringToSign =
VERB + "\n" +
Content-MD5 + "\n" +
Content-Type + "\n" +
Date + "\n" +
CanonicalizedHeaders +
CanonicalizedResource;

Table Storage:

StringToSign = 
Date + "\n" +
CanonicalizedResource

When comparing the 2 schemes, it begins to make sense why most chose to use Shared Key Lite

Which parameter must be filled, depends heavily on context. E.g. Date, while it must be set in every Table request, there is some Blob requests where it must not be set.

Canonicalized Headers

Just take all the header starting with x-ms-, sort them and concatenate them separated by \n.

Exmaple(taken from the Azure Storage documentation):

 x-ms-date:Sun, 20 Sep 2009 20:36:40 GMT\nx-ms-meta-m1:v1\nx-ms-meta-m2:v2\n

Canonicalized Resources

Canonicalized resources is form the following way:

Canonicalized resource = /account/resource

Example:

For this request

GET https://myaccount.table.core.windows.net/Tables HTTP/1.1

The canonicalized resource will be /myaccount/Tables

Query parameters must not be included. Unless you make following request(taken from documentation):

GET https://<account-name>.table.core.windows.net/?restype=service&comp=properties HTTP/1.1

Here the canonicalized resource will be /myaccount/?comp=properties

Calculating the signature

Here is the Azure Storage REST API documentation pretty weak. 2 things it misses are, when using HMAC you need to supply a key and a message. In context of making requests to the Azure Storage REST API, the key is either the Primary og Secondary key, which can be obtain from the Azure portal. The message is the StringToSign defined earlier in this post.

Also, the Primary and Secondary key which is found on the Azure portal are base64 encoded, you need to decode them, in order to be able to use them.

So what the documentation states as

Signature=Base64(HMAC-SHA256(UTF8(StringToSign))) 

Is in reality


Signature=Base64(HMAC-SHA256(UTF8(Debase64(key)),UTF8(StringToSign))) 

Where key is either the Primary or Secondary key.

And this is all for Authorization.

Wrestling the Azure Storage REST API - Part 1

Motivation

With Azure SDKs for a wide variety of programming languages, why should anybody want to learn about the Azure Storage REST API? 

Maybe there is no SDK for your favourite language, which was my case. Maybe the official SDK do not support the latest API version, which could mean it is not possible to communicate with JSON in Table Storage. Maybe you are just courios.

This blog post is based on my work on GoHaveAzureStorage, and hopefully you will also gain from challenges I have had the Azure Storage.

Request break down 

A REST call looks like this:

GET https://myaccount.table.core.windows.net/Tables HTTP/1.1

This request is used to get all tables for an storage account. There is 2 mandatory http header fields, and an additional optional which I recommend, which you must send for every request to make it work. They are 
  • x-ms-date       - time for the request.
  • x-ms-version  - Which API version is the request targeting
  • authorization  - Which is a security digest
The first 2 header will be explained in this post, while the Authorization will be explained in part 2.

The URL

First the easy part. It is possible to use either HTTP or HTTPS, else it is more or less straight forward.

The x-ms-date header field

This field is used by Azure for validation and authorization. A valid request must be maximum 15 minutes old, and it must not be dated in the future. It can be expressed as:

current time =< x-ms-date < current time - 15

Pro tip

As it is close to impossible to be complete time synchronized with Azure, it is recommend to substract a few minutes from current time, when sending request.

One important last ting, Azure only understands time in RFC1123 format and GMT +0

If you have: Thu, 12 Feb 2015 21:16:45 UTC in a +1 time zone
It must converted to: Thu, 12 Feb 2015 20:16:45 GMT

The x-ms-version header field (Pro tip)

This an optional field, but you should prefer to set it, or else you will hit an earlier version of the Azure Storage API. you might experience challenges with JSON in table storage or with Shared Access Keys if not using the latest version.

The versions are defined as date, The date which the API version is released. I'm not sure wether this a good solution, because I find dates hard to remember after a while compared to versions. So I have to look up once in a while here: https://msdn.microsoft.com/en-us/library/azure/dd894041.aspx