Monday, March 2, 2020

Adding Application Insights to a ASP.NET Core website

Application Insights may be an excellent resource for your applications and services running on Azure. Read to understand why.
Photo by SOCIAL.CUT on Unsplash

On a previous post, we discussed how to build and deploy WebJobs on Azure. Given that today it's common to have multiple instances of our apps and services (like WebJobs) running at the same time on the cloud, it's important to plan and carefully implement our logging strategy. Turns out that on Azure, the best way to do so is by using Application Insights.

On this post we will learn:
  • What's Application Insights (AppInsights)
  • What do we gain by using AppInsights
  • How to add AppInsights to an ASP.NET Core website
  • How to create and configure AppInsights on Azure
  • How to send telemetry to our Azure AppInsights instance
  • How to interpret AppInsights data on Azure

What is Application Insights

So first, let's understand what's Application Insights. According to Microsoft:
Application Insights is an extensible Application Performance Management (APM) service for developers and DevOps professionals. Use it to monitor your live applications. It will automatically detect performance anomalies, and includes powerful analytics tools to help you diagnose issues and to understand what users actually do with your app. It's designed to help you continuously improve performance and usability. It works for apps on a wide variety of platforms including .NET, Node.js and Java EE, hosted on-premises, hybrid, or any public cloud.

Services provided by AppInsights

Below I list some services provided by AppInsigths:
  • HTTP request rates, response times, and success rates
  • Dependency (HTTP & SQL) call rates, response times, and success rates
  • Exception traces from both server and client
  • Diagnostic log traces
  • Page view counts, user and session counts, browser load times, and exceptions
  • AJAX call rates, response times, and success rates
  • Server performance counters
  • Custom client and server telemetry
  • Segmentation by client location, browser version, OS version, server instance, custom dimensions, and more
  • Availability tests
Along with the preceding, there are associated diagnostic and analytics tools available for alerting and monitoring with various different customizable metrics. With its own query language and customizable dashboards, Application Insights is an excellent tool for any cloud service.

Why AppInsights?

Let's review then why and when should we use AppInsights. I usually recommend if:
  • Your project runs on the cloud
  • You run services on Azure
  • You want to gain insights on how your application runs on the cloud
  • You want to consolidate telemetry
  • You're building an App Service, Cloud Service, WebJob or Azure Functions
  • You have a moderately modern application
Desktop apps could also benefit from AppInsights this but I'd only recommend it if the developers want to learn from the telemetry submitted by their users. Remember, that an active connection should exist for the data to be pushed to Azure.

How does Application Insights work?

Once plugged to your application, AppInsights will send automatic and custom telemetry to Azure and will be available from multiple sources. The diagram below summarizes how it works:
Source: Microsoft Docs

Building our App

So let's now build our app. This post focus on .NET Core but you could also target .NET Framework. I will also try do do use .NET Core's CLI as much as possible because it not only helps us solidify the tool but also because I'm running .NET Core 3.0 which's still not supported yet by Visual Studio 2017.

Scaffolding a simple ASP.NET Core MVC WebApp using the CLI

So let's scaffold a simple ASP.NET MVC web app using the CLI. Open a Windows Terminal, navigate to the folder where you store your projects and type:
C:\src>dotnet new mvc -n aspnet-ai
The template "ASP.NET Core Web App (Model-View-Controller)" was created successfully.
This template contains technologies from parties other than Microsoft, see https://aka.ms/aspnetcore/3.1-third-party-notices for details.

Processing post-creation actions...
Running 'dotnet restore' on aspnet-ai\aspnet-ai.csproj...
  Restore completed in 136.74 ms for C:\src\aspnet-ai\aspnet-ai.csproj.

Restore succeeded.
In case you're insterested in knowing which options you have available for any project, use the --help flag. For example, to view the options for a React project, type:
dotnet new react --help
The --help flag can also be used to get information about different operations including scaffolding console, Windows forms, etc:
dotnet new --help

Testing our App

Runnning our app is as simple as typing dotnet run on the command line. The CLI then builds and runs our app:
C:\src\aspnet-ai>dotnet run
info: Microsoft.Hosting.Lifetime[0]
      Now listening on: https://localhost:5001
info: Microsoft.Hosting.Lifetime[0]
      Now listening on: http://localhost:5000
info: Microsoft.Hosting.Lifetime[0]
      Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
      Hosting environment: Development
info: Microsoft.Hosting.Lifetime[0]
      Content root path: C:\src\aspnet-ai
We can confirm the site is running by navigating to :

Adding AppInisights telemetry to our website

With the app running, let's add a reference to the Microsoft.ApplicationInsights NuGet package in our application. Back to the terminal, type  dotnet add package Microsoft.ApplicationInsights.AspNetCore :

C:\src\aspnet-ai>dotnet add package Microsoft.ApplicationInsights.AspNetCore
  Writing C:\Users\bruno.hildenbrand\AppData\Local\Temp\tmp3FE8.tmp
info : Adding PackageReference for package 'Microsoft.ApplicationInsights.AspNetCore' into project 'C:\src\aspnet-ai\aspnet-ai.csproj'.
info : Restoring packages for C:\src\aspnet-ai\aspnet-ai.csproj...

(...)

info : Package 'Microsoft.ApplicationInsights.AspNetCore' is compatible with all the specified frameworks in project 'C:\src\aspnet-ai\aspnet-ai.csproj'.
info : PackageReference for package 'Microsoft.ApplicationInsights.AspNetCore' version '2.12.1' added to file 'C:\src\aspnet-ai\aspnet-ai.csproj'.
info : Committing restore...
info : Writing assets file to disk. Path: C:\src\aspnet-ai\obj\project.assets.json
log  : Restore completed in 8.68 sec for C:\src\aspnet-ai\aspnet-ai.csproj.
To confirm that the reference was successfully added to our project, check the contents of ItemGroup/PackageReference  section:
C:\src\aspnet-ai>more aspnet-ai.csproj
<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>netcoreapp3.1</TargetFramework>
    <RootNamespace>aspnet_ai</RootNamespace>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.ApplicationInsights.AspNetCore" Version="2.12.1" />
  </ItemGroup>
</Project>

Initializing the AppInsights Framework

Next, we initialize AppInsights by adding the following call on your Startup class:
public void ConfigureServices(IServiceCollection services)
{
    // enable Application Insights telemetry
    services.AddApplicationInsightsTelemetry();
}

Creating an Application Insights resource

Let's now get started with AppInsights on Azure. On this section we will understand how to create an AppInsights instance on Azure and how to integrate it with our .NET Core app. The first thing to do to upload telemetry to AppInisghts in Azure is to create a resource for it. In Azure, click Create Resource and type Application Insights. Enter the requested information, for example:
After created, open your AppInsights resource and copy that instrumentation key (upper-right corner) as we'll need in the next step.

I like to create dashboards for each of my projects in Azure and also let available there the related AppInsights resources.

Setting the Instrumentation Key

Then, we need to configure a telemetry key in our appSettings.json file. Just paste the snippet below above the "Logging" section replacing <appInsights-key> with your Instrumentation Key:
"ApplicationInsights": {
    "InstrumentationKey": "<appInsights-key>"
}
Tip: You could also replace that via the command line if you're running a more capable OS 😊:
sed -i 's/<your-key>/<instrumentation-key>/' *.json
Run your application again. Your data should be available online in a couple of minutes. Let's now try to understand some of that data.

Understanding the telemetry

If everything was correctly configured, you should already see telemetry on Azure. Wondering why? It's because AppInisghts monitors a lot of things by default. To view your data, click on the Search tab of your AppInishgts resource and click Refresh:
It's also important to remember the main categories your data will fall into:
  • Request - a regular request to a resource on your site
  • Exception - an exception on your site
  • View - a custo page view 
  • Dependency - a call to an external resource (such as Sql Database, mail server, cache, etc)
  • Availability - will list availabilit requests to your app (another AppInsights feature).
Let's now create a custom telemetry to understand how it works.

Tracking Custom Events

To create custom telemetry we should use the TrackEvent method from the TelemetryClient class. TelemetryClient is required to post telemetry data to Azure. For an ASP.NET Core project, the TelemetryClient can be injected in our controller with:

Then, we use our telemetry client to post custom events with:

Reviewing our custom telemetry

In the above code we used the TrackEvent method of The TelemetryClient. That means that we're telling Azure to classify our telemetry as a Custom event. You can see in yellow This is the result of that message:
Clicking on the event shows the transaction:
And clicking on Custom Event we see more info. Note that we can also add custom metrics to our events:

Tracking Exceptions

A common solution to tracking exceptions would be leveraging the global exception handler. In .NET Core 3.0 that's done with:
Then we can review the result as shown below:

Common Questions

I think we covered the most essential bits with respect to AppInsights and ASP.NET Core web apps. Let's now review related and commonly asked questions regarding this implementation.

Suppressing Telemetry

You may be thinking that we have a lot more data than simply those Privacy requested logs I added to the code. As mentioned, AppInishgts automatically monitors lots of events in our applications. Yes, that's awesome but also brings drawbacks.We'll see in a future post how to suppress unnecessary data from our telemetry.

Where's my data?

If you don't see your data, please wait a little longer. It usually takes some time (up to 5 minutes) to get your telemetry online. Check also your Instrumentation Key. Remember, the SDK won't crash your application so debug to confirm it's working before deploying.

Deleting telemetry

You got your data on AppInsights and you realize that there's a lot there that you'd like to delete. How do we do it? We don't. Microsoft describes it on Data collection, retention, and storage in Application Insights, that the data cannot be deleted. And it makes sense. You should think of your telemetry as Event Sourcing. It's also better to have a lot than none.

But in case you really need it, a simple workaround would be creating another instance and using it.

Conclusion

On this post we reviewed how to add, configure, customize and inspect Application Insights telemetry from ASP.NET Core web applications and Azure. AppInsights is a fantastic tool to capture telemetry from your application if you're using Azure. I hope that this post demoed the most essential aspects of AppInsights including what's necessary within Azure.

What could we do next?

Looking for other ideas? Please consider:
  1. Plugging alerts on some of our events: we could configure Azure Alerts hooked into our application insights telemetry to alert on specific events. While this is out of scope for this post, I urge you to take a look at the feature. It's very nice indeed.
  2. Suppressing telemetry: you may think that you got more data than you need. Try to extend this example by suppressing some of the auto-submitted telemetry;
  3. Adding AppInsights to other types of applications: as discussed, we could also leverage AppInsights in everything that can communicate to the cloud - including our console applications, windows services and Windows Form apps.
  4. .NET Framework: this example was designed for .NET Core and won't run on .NET Framework. Despite the major changes in the API, the functionality on Azure and use of TelemetryClient class remains the same.
  5. .NET Core 3.0 and VS17: this example was written against the version 3.0 .NET Core framework and as of this post was not friendly with my Visual Studio 2017. Run it from the command line as instructed above as it will fail from VS17.

More about AppInsights

Want to know more about Application Insights? Consider reading the following articles:
  1. How to suppress Application Insights telemetry
  2. Monitor ASP.NET applications using Application Insights and Azure Alerts
  3. How to profile ASP.NET apps using Application Insights

Source Code

As always, the source code used on this post is available on GitHub.

References

See Also

About the Author

Bruno Hildenbrand