This post aims to explore a common approach for managing your distributed application message queueing using the different cloud pub/sub implementations.

At the end of this article you will be queueing and dequeueing messages to all three clouds using C# on the .Net Core platform.

The technologies to be used include:

Before we jump in let’s go through what you will need to get started:

If you are yet to create an account for these services go ahead and do this, as it only takes a couple of minutes, and should be cost free as long as you clean up resources once you are finished.

Over four parts we will:

Give me the code!

If you want to follow along great but if you just want to hack the code to pieces that’s fine too. You can grab all the code over on Github:

The skeleton solution provides everything up-to the end of this section.

Whist the complete solution gives you all the code for the three implementations minus the configuration settings based on your provisioned resources and credentials.

1 An Interface for our queue operations

Our queue types will implement the following two generic interfaces:

IQueuePublisher will be quite simple and will provide a method for us to place a message on the queue.

public interface IQueuePublisher<T> { Task SendMessage(T message); }

IQueueSubscriber will also be simple in that the Process method will be configured to dequeue one message at a time.

public interface IQueueSubscriber<T> { Task Process(); }

2 Solution setup, test harness and plumbing

At the end of this section will have the:

Solution and Project structure for each implementation

.Net core test harness

Basic console logging so we know what’s going on

This may seem like over zealous separation for a demo but having the logical components split out early, makes it easy to manage dependencies and keep things simple as we add more code into the project.

2.1 Solution Setup

For this demo create a solution called CloudPubSub and add a .Net Standard project named PubSub.Behaviours that will contain two interfaces; IQueuePublisher and IQueueSubscriber. Copying the respective contents from the previous section into each.

Next add the following .net standard projects for each platform implementation.

PubSub.AWS

PubSub.GCP

PubSub.Azure

For each of the implementation project, add the following classes

AWSQueuePublisher and AWSQueueSubscriber

GCPQueuePublisher and GCPQueueSubscriber

AzureQueuePublisher and AzureQueueSubscriber

Next make sure each of the publisher and subscriber class, both inherits from and implements the respective IQueuePublisher and IQueueSubscriber interfaces. When you are done you should have something that resembles:

and

At this point we have enough to get started and continue to get the test harness working. We will revisit these files later to update the constructors to pass in the settings and a logger.

2.2 Plumbing for configuration, message type and logging

Add a project for our configuration classes and a logging interface called CommonTypes and create the following folders

Settings

Behaviours

Messages

In the Behaviours folder create an interface called IAppLogger and add the operations below:

public interface IAppLogger { void LogError(Exception ex); void LogWarning(string message); void LogMessage(string message); }

In the Settings folder create the following three class files:

AWSSettings

AzureSettings

GCPSettings

Don’t worry about the contents for now we will update them later as we progress.

Finally, in the messages folder create a class called GameState, which we will use as our message container for queueing and dequeueing as we progress.

public class GameState { public string PlayerId { get; set; } public int Health { get; set; } public int CurrentLevel { get; set; } public Dictionary Inventory { get; set; } public string GameId { get; set; } }

Your solution should now look something like this:

2.3 Prepare Constructors

Next we will update the constructors of the queue publishers and subscribers to accept a settings type, and the IAppLogger interface created earlier. Make the changes listed below to the respective files

AWSQueuePublisher and AWSQueueSubscriber

GCPQueuePublisher and GCPQueueSubscriber

AzureQueuePublisher and AzureQueueSubscriber

AWSQueuePublisher and AWSQueueSubscriber:

private IAppLogger _appLogger; private AWSSettings _awsSettings; public AWSQueueX(IAppLogger appLogger, AWSSettings awsSettings) { _appLogger = appLogger; _awsSettings = awsSettings; }

GCPQueuePublisher and GCPQueueSubscriber:

private IAppLogger _appLogger; private GCPSettings _gcpSettings; public GCPQueueX(IAppLogger appLogger, GCPSettings gcpSettings) { _appLogger = appLogger; _gcpSettings = gcpSettings; }

AzureQueuePublisher and AzureQueueSubscriber:

private IAppLogger _appLogger; private AzureSettings _azureSettings; public AzureQueueX(IAppLogger appLogger, AzureSettings azureSettings) { _appLogger = appLogger; _azureSettings = azureSettings; }

2.4 TestHarness setup

Our test harness will be a simple console application that can be run locally with configuration sourced from a local appSetting.json file.

To get started add a .Net Core Console Application to the solution and name it TestHarness.

Next add an appSettings.json file and paste in the following skeleton JSON:

{ "aws": { }, "azure": { }, "gcp": { } }

These sections will be empty for now but the placeholders will be updated as we progress.

Next add a class called ConsoleLogger and paste in the code from the following gist:

This basic logger will output basic instrumentation to the console so we can monitor activity.

Next we need to add in some support for deserialization of the appSettings.json file as we will be using typed configurations classes in the code.

In the “Manage Nuget Packages” find and add the following packages to the TestHarness project:

Extensions.Configuration

Extensions.Configuration.Binder

Extensions.Configuration.FileExtensions

Extensions.Configuration.Json

When you are finished your installed packages should look like this.

Your project solution should now look something like this:

Next we need to update Program.cs with some code to load our [empty] config from the appSettings.json file, and check that our logger is alive.

Copy the code from the following gist into Program.cs

Resolve any references that may be required and Hit F5, and you should see the following:

And that’s it! All our plumbing is complete time for the fun stuff!

If you want the code for this skeleton app you can get it from GitHub here:

Proceed to part two where we will get started with the AWS Implementation.

If you have any queries / problems / feedback with this part please comment below.