Tuesday, December 1, 2020

Distributed caching in ASP.NET Core using Redis, MongoDB and Docker

Redis is the world's most popular caching database. Let's review how to implement distributed caching in ASP.NET Core using Redis, MongoDB and Docker Compose.
Photo by Christian Nielsen on Unsplash

One of the things that every modern website needs is caching. After all, we don't want to be alerted at 2AM being informed that our services are down because we had a spike in usage which our databases couldn't handle.

One common solution to reducing the stress in our applications is placing a fast caching service between our website and our database. Modern caching implementations include requirements around decreasing response time, distributed caching (sharing the same cache between multiple web instances) and cost reduction. Most implementations today use Redis (a super-fast an in-memory key–value database) as a cache service sitting in front of a database of choice.

On this post we will implement a fictional ASP.NET Core e-commerce website using MongoDB as database and Redis as a cache service, both running on Docker with Docker Compose so that we can understand how it all works together.

On this post we will:
  • Scaffold an ASP.NET Core website
  • Implement a catalog service using MongoDB
  • Implement distributed caching using Redis
  • Run our dependencies using Docker Compose
  • Setup Redis Commander and Mongo Express to view/manage our services

Setting up an ASP.NET Core website

Let's quickly scaffold an ASP.NET Core website using the command line with:
dotnet new mvc -n AspNetDistributedCaching
Then, add the below configuration to your appsettings.json file:
  "Mongo": {
    "ConnectionString": "mongodb://localhost:27017",
    "Db": "catalog",
    "Collection": "products"
  },
  "Redis": {
    "Configuration": "localhost",
    "InstanceName": "web"
  }
Next, add the config classes and bind these configs. In case you missed, feel free to review how configurations work in ASP.NET Core 3.1 projects on a previous article.

Setting up dependencies

Let's now setup our dependencies: Redis, MongoDB and the management interfaces  Redis Commander and Mongo Express. Despite sounding complicated, it's actually very simple if we use the right tools: Docker and Docker Compose.

Docker Compose 101

Without much extension, let me briefly re-introduce Docker Compose:
Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration. See the list of features.
Setting Compose in your project is actually very simple. For starters, paste this on a docker-compose.yml file on the root of the project. We'll add the services and their respective configurations next:
version: '3.7'
services:
  # we'll add our services on the next steps

Configuring our MongoDB instance

Next, let's configure MongoDB. Paste the snippet below at the bottom of your docker-compose.yml file. It will instruct Compose to init a Mongo instance called catalog-db and initialize the catalog database from the ./db.js file:
  catalog-db:
    image: mongo
    environment:
      # MONGO_INITDB_ROOT_USERNAME: root
      # MONGO_INITDB_ROOT_PASSWORD: todo
      MONGO_INITDB_DATABASE: catalog
    volumes:
    - .db.js:/docker-entrypoint-initdb.d/db.js:ro
    expose:
      - "27017"     ports:
      - "3301:27017"

Configuring our Redis instance

As with MongoDB, let's now setup our Redis cache. Paste this at the bottom of your docker-compose.yml file:
  redis:
    image: redis:6-alpine
    expose:
      - "6379"
    ports:
      - "6379:6379"

Configuring  the Management interfaces

Let's now setup management interfaces for Redis - Redis Commander and Mongo - Mongo Express to access our resources (I'll show later how do you use them). Again, paste the below on your docker-compose.yml file:
  # Mongo Express: tool to manage our Mongo database
  mongo-express:
    image: mongo-express
    restart: always
    ports:
      - "8011:8081"
    environment:
      - ME_CONFIG_MONGODB_SERVER=catalog-db
      # MONGO_INITDB_ROOT_USERNAME: root
      # MONGO_INITDB_ROOT_PASSWORD: todo
    depends_on:
      - catalog-db

  # Redis Commander: tool to manage our Redis container from localhost
  redis-commander:
    image: rediscommander/redis-commander:latest
    environment:
      - REDIS_HOSTS=redis
    ports:
      - "8013:8081"
    depends_on:
      - redis

Querying Catalog Data

Obviously in order to cache the data, we should have it first. So let's implement a simple MongoDB wrapper using the Repository Pattern which I'll call CatalogRepository. Its interface looks like:
public interface ICatalogRepository
{
    Task<IList<Category>> GetCategories();
    Task<Category> GetCategory(string slug);
    Task<Product> GetProduct(string slug);
    Task<IList<Product>> GetProductsByCategory(string slug);
}
For the concrete implemtation, don't forget to add the MongoDB.Driver NuGet Package. I show a simple query below. To view it in full, check this demo's repo:
public async Task<IList<Category>> GetCategories()
{
    var c = _db.GetCollection<Category>("categories");
    return (await c.FindAsync(new BsonDocument())).ToList();
}
Next, let's set up DI for this guy using ASP.NET's DI framework:
services.AddTransient<ICatalogRepository, CatalogRepository>();
To view the code in full, check it on this demo's repo at github.com/hd9/aspnet-distributed-caching.

Caching Catalog Data

With the repository working, let's implement the caching. I'll divide this task in:
  1. setting up Redis with distributed caching
  2. implementing a Service class and
  3. adding the caching logic to the service class.

Setting up the Redis Initialization

First, add NuGet references to Microsoft.Extensions.DependencyInjection. and Microsoft.Extensions.Caching.StackExchangeRedis. Then add the add a call to services.AddStackExchangeRedisCache in ConfigureServices:
services.AddStackExchangeRedisCache(o =>
{
    o.Configuration = cfg.Redis.Configuration;
    o.InstanceName = cfg.Redis.InstanceName;
});

Implementing a Service Class

The next part consists of implementing the caching logic. Such logic will live in CatalogService which implements the service design pattern and abstracts from the Controller either the repository and the caching implementations. Its first important part is the constructor which looks like this:
public CatalogSvc(
    ICatalogRepository repo,
    IDistributedCache cache)
{
    _repo = repo;
    _cache = cache;
}

Adding the caching logic

Then, every method of the above interface are similarly implemented. For example, the GetCategories method that feeds the landing page looks like this:
public async Task<IList<Category>> GetCategories()
{
    return await GetFromCache<IList<Category>>(
        "categories",
        "*",
        async () => await _repo.GetCategories());
}
And our GetFromCache private method that generalizes caching logic for our catalog service looks like:
private async Task<TResult> GetFromCache<TResult>(
    string key,
    string val,
    Func<Task<object>> func)
{
    var cacheKey = string.Format(_keyFmt, key, val);
    var data = await _cache.GetStringAsync(cacheKey);

    if (string.IsNullOrEmpty(data))
    {
        data = JsonConvert.SerializeObject(await func());

        await _cache.SetStringAsync(
            cacheKey,
            data);
    }

    return JsonConvert.
        DeserializeObject<TResult>(data);
}
The interesting bits of the above piece of code is that before searching the database (which's abstracted by a Func<> parameter), it searches the cache with GetStringAsync. If it finds, it returns the cached data cast as the provided type (TResult) by deserializing it from its string value. In case the cache key is not present, it will invoke the target function and cache its result as a Json string in the Redis cache.

To use it, first we wire it up to the DI container:
services.AddTransient<ICatalogSvc, CatalogSvc>();
So we can properly inject it and use it in our controllers:
public HomeController(ICatalogSvc svc)
{
    _svc = svc;
}

Management Interfaces

To finish up, let's quickly review how to access the management interfaces for Redis and Mongo Express.

Accessing Mongo Express

To view/modify your catalog, you can access Mongo Express at: http://localhost:8011/. I changed from the original port 8081 to 8011 since many services run on that port and if it was your case, Compose would fail. But feel free to change that configuration on your docker-compose.yml file. As previously mentioned, this database is auto-initialized from the db.js file by Docker Compose. Here's a quick glance of Mongo Express displaying our catalog data:

Accessing Redis Commander

Redis Commander is a frontend for vieweing and managing Redis. On this demo I run it on http://localhost:8011/. As previously, feel free to change the port on your docker-compose.yml file. Here's Redis Commander showing our cached data:

Running the Services

The last part is to describe how to run the services. As a .NET developer, you're already used to debug and run your solutions with Visual Studio - the same applies here. The only thing that remains is how to run the dependencies? As mentioned, it's as simple as running the below command from the project's root with Docker Compose:
docker-compose up
You should see the services starting in the backend similar to this:
To shutdown, run:
docker-compose down
Finally, to remove everything, run:
docker-compose down -v

Final Thoughts

On this article we reviewed how to use Redis, a super-fast key-value document database in front of MongoDB database serving as a distributed service. On this example we still leveraged Docker and Docker Compose to simplify the setup and the initialization of our project so we could get our application running and test it as quickly as possible.

Redis is one of the world's most used and loved databases and a very common option for caching. I hope you also realized how Docker and Docker Compose help developers by simplifying rebuilding complex environments like this.

Source Code

As always, the source code for this article is available on my GitHub.

References

Posted by
Labels: , , , , , , , , , , , , , ,

About the Author

Bruno Hildenbrand      
Principal Architect, HildenCo Solutions.