Monday, March 16, 2020

How to suppress Application Insights telemetry

Application Insights is an excellent tool to capture telemetry for your application but it may be too verbose. On this post we will review how to exclude unnecessary requests.
Photo by Michael Dziedzic on Unsplash

On a previous post we learned how to add Application Insights telemetry to ASP.NET applications. Turns out that we also saw that a lot of unnecessary telemetry was being sent. How to filter some of that telemetry?

On this post we will explain how to suppress telemetry by using AppInsights' DependencyTelemetry and ITelemetryProcessor.

Understanding telemetry pre-processing

You can write and configure plug-ins for the Application Insights API to customize how telemetry can be enriched and processed before it's sent to the Application Insights service in four different ways:
  1. Sampling - reduces the volume of telemetry without affecting statistics. Keeps together related data points so we can navigate between points when diagnosing a problem.
  2. Filtering with Telemetry Processors - filters out telemetry before it is sent to the server. For example, you could reduce the volume of telemetry by excluding requests from robots. Filtering is a more basic approach to reducing traffic than sampling. It allows you more control over what is transmitted, but affects your statistics.
  3. Telemetry Initializers - lets you add or modify properties to any telemetry sent from your app. For example, you could add calculated values; or version numbers by which to filter the data in the portal.

Suppressing Dependencies

So let's create a custom filter that will suppress some of that telemetry. For example, below I show all the telemetry that was automatically sent from my application to Azure with just one line of code. Our code will suppress the data shown in yellow.

Creating a Telemetry Processor

There are different types of telemetry filters being RequestTelemetry and DependencyTelemetry the most common. These classes contain properties that are useful for our logic. So let's create a filter that excludes requests to favicon.ico, bootstrap, jquery, site.css and site.js.

To get rid of some of these requests, we will implement a custom filter that I named SuppressStaticResourcesFilter to ignore requests those static resources. Our telemetry processor should implement the interface ITelemetryProcessor. Note the logic to exclude requests in the Process method:

Registering a Telemetry Processor

Now, we just need to wire it up on the initialization of our app. As stated on this document, the initialization is different for ASP.NET Core and ASP.NET MVC. Let's take a look at each of them.

The registration of a telemetry processor in ASP.NET Core is done in Startup.cs:
Configuring a telemetry processor on ASP.NET is done in Global.asax:
Wait a couple of minutes for the telemetry to show up online. You should no longer see those requests targeting static resources:

How many filters?

Well, that's up to you. You could create as many filters as you want. The only change would be on the initialization. For example, assuming For ASP.NET Core web apps, it would look like:
public void ConfigureServices(IServiceCollection services)
For classic ASP.NET MVC Web, the initialization would be:
var builder = TelemetryConfiguration.Active.DefaultTelemetrySink.TelemetryProcessorChainBuilder;
builder.Use((next) => new SuppressStaticResourcesFilter(next));
builder.Use((next) => new YourOtherFilter1(next));
builder.Use((next) => new YourOtherFilter2(next));

Enabling based on LogLevel

To finish up, we could  extend the solution above by enabling the telemetry processors based on your LogLevel. For example, the code below wouldn't register processors to suppress data if LogLevel == debug:
public void ConfigureServices(IServiceCollection services)
     if (logLevel != "debug")


On this post we extended the discussion on How to add Application Insights telemetry to ASP.NET websites and reviewed how to suppress some of that telemetry. I hope it's clear how to register your telemetry processor and how/when to use RequestTelemetry and DependencyTelemetry.

If you're using Azure, AppInsights may be a valuable asset for your organization as it provides a simple, fast, centralized and customizable service to access your logs. For more information about AppInisights, check the official documentation.


More about AppInsights

Want to know more about Application Insights? Consider reading previous articles on the same topic:
  1. Adding Application Insights telemetry to your ASP.NET Core website
  2. Monitor ASP.NET applications using Application Insights and Azure Alerts
  3. How to profile ASP.NET apps using Application Insights

Source Code

The source code used on this article is available on GitHub on the telemetry branch.

See Also

For more posts about .NET Core, please click here.

Monday, March 2, 2020

Adding Application Insights to a ASP.NET Core website

Application Insights may be an excellent resource for your applications and services running on Azure. Read to understand why.
Photo by SOCIAL.CUT on Unsplash

On a previous post, we discussed how to build and deploy WebJobs on Azure. Given that today it's common to have multiple instances of our apps and services (like WebJobs) running at the same time on the cloud, it's important to plan and carefully implement our logging strategy. Turns out that on Azure, the best way to do so is by using Application Insights.

On this post we will learn:
  • What's Application Insights (AppInsights)
  • What do we gain by using AppInsights
  • How to add AppInsights to an ASP.NET Core website
  • How to create and configure AppInsights on Azure
  • How to send telemetry to our Azure AppInsights instance
  • How to interpret AppInsights data on Azure

What is Application Insights

So first, let's understand what's Application Insights. According to Microsoft:
Application Insights is an extensible Application Performance Management (APM) service for developers and DevOps professionals. Use it to monitor your live applications. It will automatically detect performance anomalies, and includes powerful analytics tools to help you diagnose issues and to understand what users actually do with your app. It's designed to help you continuously improve performance and usability. It works for apps on a wide variety of platforms including .NET, Node.js and Java EE, hosted on-premises, hybrid, or any public cloud.

Services provided by AppInsights

Below I list some services provided by AppInsigths:
  • HTTP request rates, response times, and success rates
  • Dependency (HTTP & SQL) call rates, response times, and success rates
  • Exception traces from both server and client
  • Diagnostic log traces
  • Page view counts, user and session counts, browser load times, and exceptions
  • AJAX call rates, response times, and success rates
  • Server performance counters
  • Custom client and server telemetry
  • Segmentation by client location, browser version, OS version, server instance, custom dimensions, and more
  • Availability tests
Along with the preceding, there are associated diagnostic and analytics tools available for alerting and monitoring with various different customizable metrics. With its own query language and customizable dashboards, Application Insights is an excellent tool for any cloud service.

Why AppInsights?

Let's review then why and when should we use AppInsights. I usually recommend if:
  • Your project runs on the cloud
  • You run services on Azure
  • You want to gain insights on how your application runs on the cloud
  • You want to consolidate telemetry
  • You're building an App Service, Cloud Service, WebJob or Azure Functions
  • You have a moderately modern application
Desktop apps could also benefit from AppInsights this but I'd only recommend it if the developers want to learn from the telemetry submitted by their users. Remember, that an active connection should exist for the data to be pushed to Azure.

How does Application Insights work?

Once plugged to your application, AppInsights will send automatic and custom telemetry to Azure and will be available from multiple sources. The diagram below summarizes how it works:
Source: Microsoft Docs

Building our App

So let's now build our app. This post focus on .NET Core but you could also target .NET Framework. I will also try do do use .NET Core's CLI as much as possible because it not only helps us solidify the tool but also because I'm running .NET Core 3.0 which's still not supported yet by Visual Studio 2017.

Scaffolding a simple ASP.NET Core MVC WebApp using the CLI

So let's scaffold a simple ASP.NET MVC web app using the CLI. Open a Windows Terminal, navigate to the folder where you store your projects and type:
C:\src>dotnet new mvc -n aspnet-ai
The template "ASP.NET Core Web App (Model-View-Controller)" was created successfully.
This template contains technologies from parties other than Microsoft, see for details.

Processing post-creation actions...
Running 'dotnet restore' on aspnet-ai\aspnet-ai.csproj...
  Restore completed in 136.74 ms for C:\src\aspnet-ai\aspnet-ai.csproj.

Restore succeeded.
In case you're insterested in knowing which options you have available for any project, use the --help flag. For example, to view the options for a React project, type:
dotnet new react --help
The --help flag can also be used to get information about different operations including scaffolding console, Windows forms, etc:
dotnet new --help

Testing our App

Runnning our app is as simple as typing dotnet run on the command line. The CLI then builds and runs our app:
C:\src\aspnet-ai>dotnet run
info: Microsoft.Hosting.Lifetime[0]
      Now listening on: https://localhost:5001
info: Microsoft.Hosting.Lifetime[0]
      Now listening on: http://localhost:5000
info: Microsoft.Hosting.Lifetime[0]
      Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
      Hosting environment: Development
info: Microsoft.Hosting.Lifetime[0]
      Content root path: C:\src\aspnet-ai
We can confirm the site is running by navigating to :

Adding AppInisights telemetry to our website

With the app running, let's add a reference to the Microsoft.ApplicationInsights NuGet package in our application. Back to the terminal, type  dotnet add package Microsoft.ApplicationInsights.AspNetCore :

C:\src\aspnet-ai>dotnet add package Microsoft.ApplicationInsights.AspNetCore
  Writing C:\Users\bruno.hildenbrand\AppData\Local\Temp\tmp3FE8.tmp
info : Adding PackageReference for package 'Microsoft.ApplicationInsights.AspNetCore' into project 'C:\src\aspnet-ai\aspnet-ai.csproj'.
info : Restoring packages for C:\src\aspnet-ai\aspnet-ai.csproj...


info : Package 'Microsoft.ApplicationInsights.AspNetCore' is compatible with all the specified frameworks in project 'C:\src\aspnet-ai\aspnet-ai.csproj'.
info : PackageReference for package 'Microsoft.ApplicationInsights.AspNetCore' version '2.12.1' added to file 'C:\src\aspnet-ai\aspnet-ai.csproj'.
info : Committing restore...
info : Writing assets file to disk. Path: C:\src\aspnet-ai\obj\project.assets.json
log  : Restore completed in 8.68 sec for C:\src\aspnet-ai\aspnet-ai.csproj.
To confirm that the reference was successfully added to our project, check the contents of ItemGroup/PackageReference  section:
C:\src\aspnet-ai>more aspnet-ai.csproj
<Project Sdk="Microsoft.NET.Sdk.Web">

    <PackageReference Include="Microsoft.ApplicationInsights.AspNetCore" Version="2.12.1" />

Initializing the AppInsights Framework

Next, we initialize AppInsights by adding the following call on your Startup class:
public void ConfigureServices(IServiceCollection services)
    // enable Application Insights telemetry

Creating an Application Insights resource

Let's now get started with AppInsights on Azure. On this section we will understand how to create an AppInsights instance on Azure and how to integrate it with our .NET Core app. The first thing to do to upload telemetry to AppInisghts in Azure is to create a resource for it. In Azure, click Create Resource and type Application Insights. Enter the requested information, for example:
After created, open your AppInsights resource and copy that instrumentation key (upper-right corner) as we'll need in the next step.

I like to create dashboards for each of my projects in Azure and also let available there the related AppInsights resources.

Setting the Instrumentation Key

Then, we need to configure a telemetry key in our appSettings.json file. Just paste the snippet below above the "Logging" section replacing <appInsights-key> with your Instrumentation Key:
"ApplicationInsights": {
    "InstrumentationKey": "<appInsights-key>"
Tip: You could also replace that via the command line if you're running a more capable OS 😊:
sed -i 's/<your-key>/<instrumentation-key>/' *.json
Run your application again. Your data should be available online in a couple of minutes. Let's now try to understand some of that data.

Understanding the telemetry

If everything was correctly configured, you should already see telemetry on Azure. Wondering why? It's because AppInisghts monitors a lot of things by default. To view your data, click on the Search tab of your AppInishgts resource and click Refresh:
It's also important to remember the main categories your data will fall into:
  • Request - a regular request to a resource on your site
  • Exception - an exception on your site
  • View - a custo page view 
  • Dependency - a call to an external resource (such as Sql Database, mail server, cache, etc)
  • Availability - will list availabilit requests to your app (another AppInsights feature).
Let's now create a custom telemetry to understand how it works.

Tracking Custom Events

To create custom telemetry we should use the TrackEvent method from the TelemetryClient class. TelemetryClient is required to post telemetry data to Azure. For an ASP.NET Core project, the TelemetryClient can be injected in our controller with:

Then, we use our telemetry client to post custom events with:

Reviewing our custom telemetry

In the above code we used the TrackEvent method of The TelemetryClient. That means that we're telling Azure to classify our telemetry as a Custom event. You can see in yellow This is the result of that message:
Clicking on the event shows the transaction:
And clicking on Custom Event we see more info. Note that we can also add custom metrics to our events:

Tracking Exceptions

A common solution to tracking exceptions would be leveraging the global exception handler. In .NET Core 3.0 that's done with:
Then we can review the result as shown below:

Common Questions

I think we covered the most essential bits with respect to AppInsights and ASP.NET Core web apps. Let's now review related and commonly asked questions regarding this implementation.

Suppressing Telemetry

You may be thinking that we have a lot more data than simply those Privacy requested logs I added to the code. As mentioned, AppInishgts automatically monitors lots of events in our applications. Yes, that's awesome but also brings drawbacks.We'll see in a future post how to suppress unnecessary data from our telemetry.

Where's my data?

If you don't see your data, please wait a little longer. It usually takes some time (up to 5 minutes) to get your telemetry online. Check also your Instrumentation Key. Remember, the SDK won't crash your application so debug to confirm it's working before deploying.

Deleting telemetry

You got your data on AppInsights and you realize that there's a lot there that you'd like to delete. How do we do it? We don't. Microsoft describes it on Data collection, retention, and storage in Application Insights, that the data cannot be deleted. And it makes sense. You should think of your telemetry as Event Sourcing. It's also better to have a lot than none.

But in case you really need it, a simple workaround would be creating another instance and using it.


On this post we reviewed how to add, configure, customize and inspect Application Insights telemetry from ASP.NET Core web applications and Azure. AppInsights is a fantastic tool to capture telemetry from your application if you're using Azure. I hope that this post demoed the most essential aspects of AppInsights including what's necessary within Azure.

What could we do next?

Looking for other ideas? Please consider:
  1. Plugging alerts on some of our events: we could configure Azure Alerts hooked into our application insights telemetry to alert on specific events. While this is out of scope for this post, I urge you to take a look at the feature. It's very nice indeed.
  2. Suppressing telemetry: you may think that you got more data than you need. Try to extend this example by suppressing some of the auto-submitted telemetry;
  3. Adding AppInsights to other types of applications: as discussed, we could also leverage AppInsights in everything that can communicate to the cloud - including our console applications, windows services and Windows Form apps.
  4. .NET Framework: this example was designed for .NET Core and won't run on .NET Framework. Despite the major changes in the API, the functionality on Azure and use of TelemetryClient class remains the same.
  5. .NET Core 3.0 and VS17: this example was written against the version 3.0 .NET Core framework and as of this post was not friendly with my Visual Studio 2017. Run it from the command line as instructed above as it will fail from VS17.

More about AppInsights

Want to know more about Application Insights? Consider reading the following articles:
  1. How to suppress Application Insights telemetry
  2. Monitor ASP.NET applications using Application Insights and Azure Alerts
  3. How to profile ASP.NET apps using Application Insights

Source Code

As always, the source code used on this post is available on GitHub.


See Also

About the Author

Bruno Hildenbrand      
Principal Architect, HildenCo Solutions.