Monday, November 2, 2020

Async Request/Response with MassTransit, RabbitMQ, Docker and .NET core

Let's review how to implement an async resquest/response exchange between two ASP.NET Core websites via RabbitMQ queues using MassTransit
Photo by Pavan Trikutam on Unsplash

Undoubtedly the most popular design pattern when writing distributed application is Pub/Sub. Turns out that there's another important design pattern used in distributed applications not as frequently mentioned, that can also be implemented with queues: async requests/responses. Async requests/responses are very useful and widely used to exchange data between microservices in non-blocking calls, allowing the requested service to throttle incoming requests via a queue preventing its own exhaustion.

On this tutorial, we'll implement an async request/response exchange between two ASP.NET Core websites via RabbitMQ queues using MassTransit. We'll also wire everything up using Docker and Docker Compose.

On this post we will:
  • Scaffold two ASP.NET Core websites
  • Configure each website to use MassTransit to communicate via a local RabbitMQ queue
  • Explain how to write the async request/response logic
  • Run a RabbitMQ container using Docker
  • Test and validate the results

Understanding MassTransit Async Requests

If you understand how to wire everything up, setting up async request/response with MassTransit is actually very simple. So before getting our hands into the code, let's review the terminology you'll need to know:
  • Consumer: a class in your service that'll respond for requests (over a queue on this case);
  • IRequestClient<T>: the interface we'll have to implement to implement the client and invoke async requests via the queue;
  • ReceiveEndpoint: a configuration that we'll have to setup to enable our Consumer to listen and respond to requests;
  • AddRequestClient: a configuration that we'll have to setup to allow our own async request implementation;
Keep that info in mind as we'll use them in the following sections.

Creating our Project

Let's quickly scaffold two ASP.NET Core projects by using the dotnet CLI with:
dotnet new mvc -o RequestSvc
dotnet new mvc -o ResponseSvc

Adding the Dependencies

The dependencies we'll need today are:

Adding Configuration

The configuration we'll need  is also straightforward. Paste this in your RequestSvc/appsettings.json:
"MassTransit": {
    "Host": "rabbitmq://localhost",
    "Queue": "requestsvc"
}
And this in your ResponseSvc/appsettings.json:
"MassTransit": {
    "Host": "rabbitmq://localhost",
    "Queue": "responsesvc"
}
Next, bind the config classes to those settings. Since I covered in detail how configurations work in ASP.NET Core 3.1 projects on a previous article I'll skip that to keep this post short. But if you need, feel free to take a break and understand that part first before you proceed.

Adding Startup Code

Wiring up MassTransit in ASP.NET DI framework is also well documented. For our solution it would look like this for the RequestSvc project:
services.AddMassTransit(x =>
{
    x.AddBus(context => Bus.Factory.CreateUsingRabbitMq(c =>
    {
        c.Host(cfg.MassTransit.Host);
        c.ConfigureEndpoints(context);
    }));
   
    x.AddRequestClient<ProductInfoRequest>();
});

services.AddMassTransitHostedService();
And like this for the  ResponseSvc project:
services.AddMassTransit(x =>
{
    x.AddConsumer<ProductInfoRequestConsumer>();

    x.AddBus(context => Bus.Factory.CreateUsingRabbitMq(c =>
    {
        c.Host(cfg.MassTransit.Host);
        c.ReceiveEndpoint(cfg.MassTransit.Queue, e =>
        {
            e.PrefetchCount = 16;
            e.UseMessageRetry(r => r.Interval(2, 3000));
            e.ConfigureConsumer<ProductInfoRequestConsumer>(context);
        });
    }));
});

services.AddMassTransitHostedService();
Stop for a second and compare the differences between both initializations. Spot the differences?

Building our Consumer

Before we can issue our requests, we have to build a consumer to handle these messages. In MassTransit's world, this is the same consumer you'd build for your regular pub/sub. For this demo, our ProductInfoRequestConsumer looks like this:
public async Task Consume(ConsumeContext<ProductInfoRequest> context)
{
    var msg = context.Message;
    var slug = msg.Slug;

    // a fake delay
    var delay = 1000 * (msg.Delay > 0 ? msg.Delay : 1);
    await Task.Delay(delay);

    // get the product from ProductService
    var p = _svc.GetProductBySlug(slug);

    // this responds via the queue to our client
    await context.RespondAsync(new ProductInfoResponse
    {
        Product = p
    });
}

Async requests

With consumer, configuration and the startup logic in place, it's time to write the request code. In essence, this is the piece of code that will mediate the async communication between the caller and the responder using a queue (abstracted obviously by MassTransit). A simple async request to a remote service using a backend queue looks like:
using (var request = _client.Create(new ProductInfoRequest { Slug = slug, Delay = timeout }))
{
    var response = await request.GetResponse<ProductInfoResponse>();
    p = response.Message.Product;
}

Running the dependencies

To run RabbitMQ, we'll use Docker Compose. Running RabbitMQ with Compose is as simple as running the below command from the src folder:
docker-compose up
If everything correctly initialized, you should expect to see RabbitMQ's logs emitted by Docker Compose on the terminal:
To shutdown Compose and RabbitMQ, either click Ctrl-C or run:
docker-compose down
Finally, to remove everything, run:
docker-compose down -v

Testing the Application

Open the project from Visual Studio 2019, and run it as debug (F5) and VS will open 2 windows - one for RequestSvc and another for ResponseSvc. RequestSvc looks like this:

Go ahead and run some queries. If you got your debugger running, it will stop in both services allowing you to validate the exchange between them. To reduce Razor boilerplate the project uses VueJS and AxiosJs so we get responses in the UI without unnecessary roundtrips.

RabbitMQ's Management Interface

The last thing worth mentioning is how to get to RabbitMQ's management interface. This project also allows you to play with RabbitMQ at http://localhost:8012. By logging in with guest | guest and clicking on the Queues tab you should see something similar to:
RabbitMQ is a powerful message-broker service. However, if you're running your applications on the cloud, I'd suggest using a fully-managed service such as Azure Service Bus since it increases the resilience of your services.

Final Thoughts

On this article we reviewed how to implement an asynchronous request/response using queues. Async resquests/responses are very useful and widely used to exchange data between microservices in non-blocking calls, allowing the resqueted service to throttle incoming requests via a queue preventing its own exhaustion. On this example we still leveraged Docker and Docker Compose to simplify the setup and the initialization of our backend services.

I hope you liked the demo and will consider using this pattern in your applications.

Source Code

As always, the source code for this article is available on my GitHub.

References

See Also

Posted by
Labels: , , , , , , , , , , , , , , , , , , ,

About the Author

Bruno Hildenbrand      
Principal Architect, HildenCo Solutions.