Communication in Windows Azure

Posted on December 6, 2010 by

5


Overview

In Windows Azure the environment is dynamic, each role instance is deployed on a different virtual machine and there is a limited control over IPs and ports. There are three main ways to communicate between instances in Windows Azure:

1. Direct communication – through WCF

2. Azure queue storage

3. Windows Azure Service Bus + Message buffers

The types of communication which should be taken into an account are the following:

1. External endpoints exposed to the web through web roles

2. Storage approaches through REST API[1]

3. Communication inside Azure

In this post, I will describe briefly each way of communication, and later hold a comparison between these three types.

Queues

A queue stores messages and makes them available to applications via the REST API. It is highly scalable and available for asynchronous message communication since the messages expiration time can be up to seven days.

There can be any number of queues in an account in the Queue service, and there is no limit to the number of messages that can be stored in a queue, but the size of each individual message can’t exceed 8KB. To communicate large object messages, the object can be put in a blob and only its URI will be sent as a message to the queue.

 
 clip_image001[3]  

Each message in the queue contains the following attributes:

Attribute Description
MessageID Identifier of the message which is created by the Queue service
PopReceipt Used for deleting or popping the message out from the queue
Visibilitytimeout represents in seconds how much time the message remains visible after it’s received by a receiving application (default is 30 seconds, maximum value is 2 hours).
Messagettl Time-to-live value in seconds.Represents how much time the message is kept by the queue service (default is 7 days).

Accessing the storage API is done through REST API through the following Uri:

<http|https>://<account name>.queue.core.windows.net/<queue name>/messages

Approaching the storage API can be done either with a wrapper class located in Microsoft.WindowsAzure.StorageClient in Windows Azure SDK or with a custom wrapper client above the REST API.

The StorageClient.dll (in the azure SDK) uses HttpWebRequest and HttpWebResponse to send REST messages over HTTP.

The main classes in StorageClient.dll are:

Class Name Description
CloudStorageAccount Used for receiving account information from the configuration file or creating an instance of the storage account objects from account parameters.
CloudQueueClient Used for receiving references to the core queue objects. The class consists of methods like GetQueueReference() and ListQueues().
CloudQueue Includes queue operations like Create (), Delete (), AddMessage(), and GetMessage().
CloudQueueMessage Represents a queue message with properties like InsertionTime, ExpirationTime, NextVisibleTime, Id etc.

Azure Service Bus/ Message Buffer

The Service Bus is built on Windows Communication Foundation (WCF) and uses standard Internet protocols. It is cloud-based, and its aim is to simplify connectivity between services and clients, even if those servers and clients exist behind different firewalls and NAT routers. It helps applications communicate with each other, regardless of network specific structure.

Another core part of the Azure service bus is the Access Control service (ACS), a publicly accessible identity and authentication management service which is used to secure the Service Bus endpoints[2].

Message Buffer is WCF queue in the cloud (although currently it does not provide all features of WCF queue) which can be consumed by non-azure components. The messages are stored in buffers for a configurable period of time, even when no service is monitoring the buffer.

Message buffers have some limitations:

• Maximum of maxMessageCount is 50

• Maximum of ExpiresAfter is 10 minutes.

• Currently, the only available action when the buffer fills up with messages is to reject the incoming messages by faulting the message back to the sender.

clip_image001[5]

WCF

WCF allows direct communication between role instances.

In this approach, roles expose an internal or external endpoint and either bind a WCF service that would listen on this endpoint (if it’s the server) or create a WCF proxy (if it’s the client).

The real Uri which should be approached or listened on, should be discovered through the RoleEnvironment.

For example:

RoleEnvironment.CurrentRoleInstance.InstanceEndpoints[“MyService”];

 
  clip_image001[7]

 

Comparison

  Queues WCF Message buffers
Designed scenario More suitable for code hosted within Azure itself.
Used for One-way (potentially disconnected) communication.
Two-way connected communication.
Between roles inside azure and endpoints
facing the internet inside a WebRole.
More specifically designed for integrating systems rather than providing a general purpose reliable messaging system. Used for One-way (potentially disconnected) communication.
Client and server can be behind different networks.
Limit on message size Each message can’t be more than 8KB in size.(However, the message data is stored in base-64 encoded form so the maximum size of actual data in the message is somewhat less than this – perhaps around 6KB) No limit on message size (configurable by MaxMessageSize parameter) Maximum size of 64 KB
Order of messages Messages can be received in any order.Messages will remain in the queue until explicitly deleted. If a message is read by one process, it will be marked as invisible to other processes for a variable time period, which defaults to 30 seconds, and can be no more than 2 hours; if the message hasn’t been deleted by then, it will be returned to the queue and will be available for processing again. Because of this behavior, there is also no guarantee that messages will be in any particular order. Configurable – possible to configure that message order is needed (parameter of ReliableSession) Messages can be received in any order.
Reliability A message may be received more than once.Supports disconnected servers. Configurable – possible to configure ReliableSessions, which guarantees a message is delivered only once.No support against disconnected servers. A message may be received more than once.Supports disconnected servers.
Messages format When using the StorageClient.dll messages sent to the Queue service can be in either text or binary format, but received messages are always in the Base64 encoded format.Serialization is done when creating the CloudQueueMessage.

It’s possible to create a wrapper layer above WCF and use the DataContract or serializable attributes.

Message can contain Any user-defined class with DataContract or serializable attribute Message can contain Any user-defined class with DataContract or serializable attribute
Payment Per GB stored & transactions Communication that does not cross a datacenter boundary does not cost money, otherwise payment is Per bandwidth. Per Connection.See WindowsAzurePricing
Server side Create a polling thread that after each PollInterval calls the GetMessage() operation periodically. ServiceHost listens for incoming requests. ServiceHost listens for incoming requests.
Client side StorageClient or any other wrapper class above the REST API Through a WCF proxy The only way to interact with Durable Message Buffers is through a RESTful interface.Can be implemented above WCF or WebHttpRequest/

Response.

Optimization Frequent message passing with a small amount of data can affect performance and cost, and it is advisable to use storage APIs to group transactions.   Payment is per connection, so optimization may be needed (same as the Azure queue)
Latency The calls will execute when the service processes the messages at some time in the future. (depends on the server’s polling interval) No latency Latency exists
Obtain results No way out of the box.Can be implemented through ResponseBuffer.

(see useful links)

Exists .(exceptions/return value) No way out of the box.Can be implemented through ResponseBuffer.

(see useful links)

 Pros and cons

Windows Azure queues

Pros:

1. The main benefit of working with queues is its high reliability of message delivery. Queues can work against disconnected servers and an item can stay in the queue for 7 days

2. Changing the “service” topology by adding or removing role instances does not interrupt the service and does not cause loss of messages

3. Support in durable services and transactions

Cons:

1. No guarantee for FIFO

2. Message type can be string or bytes (could use helper class to serialize other types, or implement a wrapper layer above WCF and use WCF serialization)

3. Message size limit is 8KB per message

4. Latency of messages which depends on the server’s side polling interval.

5. No way (out of the box) to obtain the results of the message invocation or any errors. (though it is possible to define a queued response service similar to the one described in here – http://msdn.microsoft.com/en-us/magazine/cc163482.aspx).

Message buffer

Pros:

1. Enables connecting applications behind different networks  through a relay service in the cloud

2. Single place for managing authentication  of the client and services

3. Preventing DOS attacks

Cons:

1. Buffers are limited at size – maximum 50 messages.

2. Maximum message size is 64 KB

3. Payment per transaction

4. Not Durable – the messages are stored in memory.

5. Service bus buffers are not transactional and sending or receiving messages cannot be part of a transaction

6. Cannot handle long lasting messages (messages can live up to 10 minutes)

7. No way (out of the box) to obtain the results of the message invocation or any errors.

WCF

Pros:

1. Communication that does not cross a datacenter boundary does not cost money, otherwise the payment is only for bandwidth

2. No latency for delivering messages

3. Possible to configure reliable sessions and guarantee reliability and order of messages

4. Indication for errors (receives exception)

5. Supports results for operations (return values)

6. WCF serialization

7. Allows extensibility in multiple places in the chain

Cons:

1. Does not support working against disconnected servers

2. More complex to support transaction and durable service, since it does not involve storage, but it is possible see – http://msdn.microsoft.com/en-us/magazine/cc163688.aspx

Useful links


[1] Approaches to Azure Storage, aka queues, blobs and tables are done through a REST API.

[2] ACS receives an issuer name and an issuer secret (32-bit key), if they are correctly authenticated, issues a security token (called Simple Web Token) that is then trusted by the service itself. All messages sent to and via the Service Bus must contain a valid security token.

Advertisements
Posted in: Azure