Monday, February 17, 2020

Running NServiceBus on Azure WebJobs

On this post we will learn how to build and deploy NServiceBus on Azure WebJobs.
Photo by True Agency on Unsplash

Most of us aren't there yet with microservices. But that doesn't mean we shouldn't upgrade our infrastructure to reduce costs, increase performance, enhance security, simplify deployment, and scaling using simpler/newer technologies. On this article let's review how to deploy NServiceBus on Azure WebJobs and discuss:

  • why use WebJob
  • how to upgrade your NSB endpoints as Azure WebJob;
  • the refactorings you'll need
  • how to build, debug, test and deploy our WebJob;
  • production considerations
If you're starting a new project or have a light backend, I'd recommend you to consider going serverless on Azure Functions or AWS Lambda with SQS.

Introduction

On Azure, NServiceBus is usually deployed on Cloud Services or Windows Services. The major problem with both is that, by being Platform on a Service (PAAS) they are difficult to update, secure, scale out and have to be managed separately.

Depending on your usage, migrating your NSB backend to Azure WebJobs could be a good alternative to reduce costs, maintenance and increase security. Plus, since webjobs are scaled out automatically with your Azure App Services, you would literally get autoscaling on your NSB backend for free!

Azure WebJobs

So let's start by quikcly recapping what are webjobs. According to Microsoft, webjobs are
a feature of Azure App Service that enables you to run a program or script in the same context as a web app, API app, or mobile app. There is no additional cost to use WebJobs.
In other words, a WebJob is nothing more than a script or an application run by Azure. Currently supported formats are:
  • .cmd, .bat, .exe (using Windows cmd)
  • .ps1 (using PowerShell)
  • .sh (using Bash)
  • .php (using PHP)
  • .py (using Python)
  • .js (using Node.js)
  • .jar (using Java)
As of the creation of this post, Azure still didnt' support WebJobs on App Service on Linux.

Types of WebJobs

WebJobs can be triggered and continuous. The differences are:
  • Continuous WebJobs: start immediately, can be run in parallel or be restricted to a single instance.
  • Triggered WebJobs: can be triggered manually or on a schedule and run on a single instance selected by Azure.
Since backend services like NServiceBus and MassTransit traditionally run continuously on the background, this post will focus on continuous WebJobs.

Benefits of running NServiceBus on WebJobs

So what are the benefits of transitioning our NServiceBus hosts to WebJobs? In summary, this appoach will:
  • reduce your costs as no VMs or Cloud Services are required
  • allow your backend to scale up automatically with your Azure App Service
  • eliminates your concerns about maintenance/upgrade/patch and security
  • is way simpler to deploy
  • differently than NServiceBus Host, is not being deprecated
So let's review how it works.

Migrating NServiceBus backends to WebJobs

Migrating NServiceBus backend to WebJobs couldn't be simpler. Since, NSB's official documentation does not clearly describes a migration process, let's address it here. Essentially you'll have to:
  • transform your host project in a console application
  • add a startup class and refactor the endpoint initialization
  • add a reference Microsoft.Azure.WebJobs so you can use the WebJob Api (optional)

Transforming our Host in a Console Application

The first part of our exercise requires converting our NServiceBus endpoint to a WebJob. Since WebJobs are essentially executable files we can start by simply transforming our endpoint project from a Class Library to a Console Aplication:

Referencing the WebJob package

As recommended, to leverage the Azure Api we'll have to add the Microsoft.Azure.WebJobs NuGet package to our solution. After that package is added, we'll also have to refactor our Main method to correctly initialize and shutdown our bus.

Adding a Startup class

Next, we have to add a startup class to our project. Essentially the compiler just needs a static void Main() method inside our solution so the project can be initialized. This is a simple example:
From here, not much will change. In summary, you will have to:
  • Remove the NServiceBus.Host pakage from the solution
  • Remove IWantToRunWhenEndpointStartsAndStops as it's no longer necessary
  • Refactor some of your settings because your deployment will likely change.
  • Optionally, add some sort of centralized logging like Application Insights since your backend will run in multiple instances. And having a consolidated logging infrastructure wouldn't hurt.

Potential Problems

If you updated to the 3.x series of the Microsoft.Azure.WebJobs NuGet package, you probably realized that Microsoft aligned .NET Core and the .NET Framework on this release, already preparing for .NET 5. While that's excellent news, I you may also have conflicting dependencies and build errors as the one listed below.
error CS0012: The type 'Object' is defined in an assembly that is not referenced. You must add a reference to assembly 'netstandard, Version=2.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'.
That error can be fixed by:
  1. Referencing the NETStandard.Libray NuGet package on your WebJob;
  2. Adding <Reference Include="netstandard" /> just below the <ItemGroup> section on your WebJob's csproj file.

    Building, testing and debugging the solution

    After upgrading packages, refactoring code and fixing dependencies issues, we'll now have to fix potential build errors, and assert that the unit-tests are passing for the solution solution. Since I don't expect any major issues here, guess we can move ahead and review necessary changes for debugging.

    Debugging

    Because we transformed our NServiceBus endpoint in a console app, we remain able to start it on debug as previously. However there are some important details ahead. Make sure you set your WebJob project to start when debugging. To do so, we have to configure our solution to start multiple projects at the same time by right-clicking your solution, clicking Startup scripts, selecting Multiple startup projects and setting the Action column to Start for the projects you want to start.
      Now set your Azure Storage connection string so you can debug your project with Azure.

      Running Locally

      But if you want to run the WebJob using the development connection string ("UseDevelopmentStorage=true"), you will realize that the initialization fails. WebJobs can't run with the Azure Storage Emulator: 
      System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException:
   Failed to validate Microsoft Azure WebJobs SDK Storage account.
   The Microsoft Azure Storage Emulator is not supported, please use a Microsoft Azure Storage account 
hosted in Microsoft Azure.
   at Microsoft.Azure.WebJobs.Host.Executors.StorageAccountParser.ParseAccount(String connectionString,
String connectionStringName, IServiceProvider services)
   at Microsoft.Azure.WebJobs.Host.Executors.DefaultStorageAccountProvider.set_StorageConnectionString(String value)
   at Microsoft.Azure.WebJobs.JobHostConfiguration.set_StorageConnectionString(String value)
   at ATIS.Services.Program.d__1.MoveNext() in C:\src\ATIS\ATIS.Services\Program.cs:line 49
   --- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
      So if we have a console application ready to run, why do even have to start the Jobhost at all?

      That's a common scenario in which we want to run a different logic on debug and release modes, I usually resort to preprocessor directives where I have different implementations for debug and release. The next snippet shows it on line 15:

      Going Async

      To finish, let's make code asynchronous. The two previous snippets could be refactored into  something like:

      Deployment

      Now let's discuss deployment. Three important things to note:
      1. Variable collisions - you will probably have to change or rename variables since some of them may overlap;
      2. Changes in the deployment process - to be run as a WebJob, your backend will have to be deployed with your web app;
      3. Transformations - will probably have to change some transformations so they're also available for the backend.

      Deploying your WebJob

      A continuous webjob should be deployed with your Azure App Service on App_data/jobs/continuous. Triggered jobs should go into the App_data/jobs/triggered folder. The screenshot below shows them running in my AppService:
      Another way to confirm that is by using the Azure Serial Console and cding into that folder:

        Changing the Deployment Process

        So how do we get our WebJobs on App_data/jobs/continuous? Well, that will obviously depend on how you're deploying your services. The most common deployment strategies are:
        1. ClickOnce from Visual Studio
        2. Custom PowerShell scripts
        3. Using an automated deployment tool (ex. Azure DevOps, CircleCI, AppVeyor, Octopus Deploy, etc)
        4. By hand 😢 
        Let's discuss the two most common ways: NuGet packages and PowerShell Scripts.

        NuGet Packaging

        A common way to package code is building NuGet Packages. I won't extend much into that as it's outside of the scope of this post but I want to highlight that getting our project within our NuGet package is very simple. If you're already building NuGet packages, by simply add a reference to your project on the <files> section, we're telling msbuild to package our project with our web application:

        PowerShell

        If your CI/CD supports PowerShell, we could add the below snippet in a step just before the release:
        # PowerShell Copy-Item ..\MyApp.Backend\bin\release App_Data\jobs\continuous\MyApp.Backend -force -recurse

        Post-build event

        Another alternative would be running a specific command you your post-build event. Just keep in mind that this would also slow your local builds unless if add some contitional around it:
        # xcopy xcopy /Q /Y /E /I ..\MyApp.Backend\bin\release App_Data\jobs\continuous\MyApp.Backend

        Testing Considerations

        With the deployment out of the way, let's what should be considered when testings:
        1. Performance - I didn't see any degradation performance changes but that could not be your case. Test and compare the performance of this implementation.
        2. Failures - A crashing WebJob won't crash your App Service but, have you tested edge cases?
        3. Scale - the number of instances can be different from your current setup. Can you guarantee that no racing conditions exist? 
        4. Logging - Do you need to change how your application logs its data? Are the logs centralized and easily accessible?
        5. Remoting - Because you departed Windows VMs and Cloud Services doesn't mean that you can't access the instance remotely. The Azure Serial Console is an excellent tool to manage and inspect some aspects of your job.

          Production Considerations

          Still there? So let's finish this post with some considerations about running NServiceBus on WebJobs in production. I expect you tested your application against the items highlighted on the previous section.

          I'd recommend that before going to production that you:
          • build some metrics - around the performance before deploying so you know what to expect;
          • use Azure Deployment Slots - to validate production before setting it live;
          • doubletriple-check your configuration - because it's a new deployment to a new environment and some configurations were changed, weren't they?
          • keep an eye on the logs - as we always do, right? 😊 
          • do a post-mortem after the deployment - so your team reflects on the pros/cons of this transition.

          Final Thoughts

          Migrating NServiceBus from VMs to WebJobs was a refreshing and cost-saving experience. Over time, we felt the heavy burden of managing VMs (security patches, firewalling, extra configuration, redundancies, backups, storage, vNets, etc) not to mention how difficult it is to scale them out. Because WebJobs scale out automatically with the App Service at virtually no extra cost, we definitely gained a lot with this change. Some of the positive impacts I saw were:
          • quicker deployments
          • easier to scale out
          • cheaper to run
          • more secure
          • reduced zero ops
          • simpler deployments
          • decent performance
          If you're starting a new project or have a light backend, I'd recommend you to consider going serverless on Azure Functions or AWS Lambda with SQS.

          More about NServiceBus?

          Want to read other posts about NServiceBus, please also consider:

          References

          See Also

            Posted by
            Labels: , , , , , , , , , , , , , , , ,

            About the Author

            Bruno Hildenbrand