WebJobs in Azure with .NET Core 2.1

WebJobs aren't new to Azure or .NET. There's even a default Azure WebJob template in Visual Studio 2017 for the full .NET Framework. However, a similar template for WebJobs in .NET Core is somehow missing from Visual Studio. In this post, I'm using .NET Core 2.1.

Creating a WebJob in .NET Core isn't hard, but you have to know some tricks, especially if you want to use some .NET Core goodies like logging and DI.

In this post, we're going to build a WebJob and release it to Azure using Visual Studio, the Azure portal, and VSTS.

You can find the code samples for this post on GitHub.

What are WebJobs

A WebJob is a program running in the background of an App Service. It runs in the same context as your web app at no additional cost. Maybe you need to do some hourly task or do some cleanup task every night at 1 AM. Azure Application Insights uses a WebJob to report your app's statistics.

WebJobs can be scheduled, like hourly or daily, but they can also be triggered. A trigger could be a file upload or a new message on a queue.

WebJobs vs. Functions

I've often found comparisons between WebJobs and Azure Functions. In a way, Functions are the successors to WebJobs. Functions are (usually) small pieces of code that run in Azure and are, just like WebJobs, triggered at a certain event, including an HTTP trigger.

Functions are often a great alternative to WebJobs, but if you already have a web app it could make sense to use a WebJob instead. Especially if you want to share code and/or settings between the WebJob and your web app as they run in the same context, which also makes deployment quite easy.

Creating a Storage Account

Before we continue let's take care of something first. A WebJob requires an Azure Storage Account.  I'll quickly walk you through the process of creating one.

In Azure, find "Storage Accounts" and add one. You'll have to pick a name that's unique across Azure. Other than that you can leave the defaults. We're talking about cents per GB, so don't worry about costs too much.

Once your Storage Account is ready, select it and find your "Access keys". We'll need one of the two connection strings later.

Creating a WebJob

As said, there's a WebJob template for the full .NET Framework. I recommend you check it out. Start by creating an ASP.NET Web Application and then add a new WebJob. If you try to create the WebJob right away you'll get an error saying that the project needs to be saved first (although it does create the WebJob).

We're here for the .NET Core version of a WebJob though. So start by creating an ASP.NET Core Web Application and then add a new .NET Core Console App project to your solution.

The first thing we need to do to is install the Microsoft.Azure.WebJobs package from NuGet. We should also install Microsoft.Azure.WebJobs.Extensions. Here's the catch though, the latest stable versions of these libraries have dependencies on the full .NET Framework so we're going to need version 3.0.0-beta5 (at the time of this writing), which is fully compatible with .NET Core.

Other NuGet packages we'll need are Microsoft.Extensions.Options.ConfigurationExtensions (which also gives us the Microsoft.Extensions.Options package, which we also need), Microsoft.Extensions.DependencyInjection and Microsoft.Extensions.Logging.Console. Be sure to install version 2.1.0 of these packages because there seems to be a bug in .NET Core 2.1 that prevents you from using packages with patch versions, like 2.1.1.

Join the Program

The next thing we need to do is change our Program.cs file. If you've created a WebJob using the .NET Framework template you can simply copy and paste the Program.cs file that was generated there (although you might want to change the namespace).

[code]using Microsoft.Azure.WebJobs;

namespace NetCoreWebJob.WebJob
{
// To learn more about Microsoft Azure WebJobs SDK, please see https://go.microsoft.com/fwlink/?LinkID=320976
internal class Program
{
// Please set the following connection strings in app.config for this WebJob to run:
// AzureWebJobsDashboard and AzureWebJobsStorage
private static void Main()
{
var config = new JobHostConfiguration();
if (config.IsDevelopment)
{
config.UseDevelopmentSettings();
}
var host = new JobHost(config);
// The following code ensures that the WebJob will be running continuously
host.RunAndBlock();
}
}
}

Adding Configuration and DI

So I promised you'd get all the .NET Core goodies like logging and DI. By default, a Console App doesn't have any of that, but you can add it yourself.

[code]private static void Main()
{
IServiceCollection services = new ServiceCollection();
ConfigureServices(services);

// ...
}

private static IConfiguration Configuration { get; set; }

private static void ConfigureServices(IServiceCollection services)
{
var environment = Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT");

Configuration = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
.AddJsonFile($"appsettings.{environment}.json", optional: true, reloadOnChange: true)
.AddEnvironmentVariables()
.Build();

services.AddSingleton(Configuration);
services.AddTransient<Functions, Functions>();
services.AddLogging(builder => builder.AddConsole());
}

Next, create an appsettings.json file and set the "Copy to Output Directory" property to "Copy always". The appsettings.json file should have two connection strings as mentioned in the Program.cs template file. These are the Storage Account connection strings we created earlier.

[code]{
"Logging": {
"LogLevel": {
"Default": "Warning"
}
},
"ConnectionStrings": {
"AzureWebJobsDashboard": "[your Storage Account connection string]",
"AzureWebJobsStorage": "[your Storage Account connection string]"
}
}

The next thing we need is a custom IJobActivator that can be used to inject dependencies into our classes. It needs to be set on the JobHostConfiguration in the Program class.

[code]using Microsoft.Azure.WebJobs.Host;
using Microsoft.Extensions.DependencyInjection;
using System;

namespace NetCoreWebJob.WebJob
{
public class JobActivator : IJobActivator
{
private readonly IServiceProvider services;

public JobActivator(IServiceProvider services)
{
this.services = services;
}

public T CreateInstance<T>()
{
return services.GetService<T>();
}
}
}

[code]var config = new JobHostConfiguration();
config.JobActivator = new JobActivator(services.BuildServiceProvider());

Adding a Trigger

After that, create a class and name it Functions (just like in the WebJob template). The Functions class will have the actual code for our WebJob.

Of course, we'll need to add a trigger. This is different than the full .NET Framework. After all, the template uses a static method, which makes DI impossible. Speaking of DI, notice that we've also added the Functions class itself to the DI container.

For simplicity, we'll use a TimerTrigger, which is triggered with a so-called CRON expression. This simply means it's triggered at a certain minute, hour, day, etc. In this example, it triggers every minute.

We'll also need to configure timers on the JobHostConfiguration.

[code]using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;
using System;

namespace NetCoreWebJob.WebJob
{
public class Functions
{
private readonly ILogger<Functions> logger;

public Functions(ILogger<Functions> logger)
{
this.logger = logger;
}

public void ProcessQueueMessage([TimerTrigger("* * * * *")]TimerInfo timerInfo)
{
logger.LogInformation(DateTime.Now.ToString());
}
}
}

[code]var config = new JobHostConfiguration();
config.JobActivator = new JobActivator(services.BuildServiceProvider());
config.UseTimers();

Running the example

If you did everything correctly, or if you're running my code from GitHub, you should now be able to run the Console App. If you break on exceptions or if you're watching the Output window you may notice a lot of StorageExceptions. Don't worry about them and ignore them. This is a bug in the WebJobs library and will not affect your program. It may take a minute for your trigger to go off, so have a little patience.

If you head over to your Azure Storage Account you should see two Blob Containers, "azure-jobs-host-output" and "azure-webjobs-hosts". There's quite a lot going on here, but you can just ignore it. I've found that my WebJob triggers wouldn't go off for some reason, deleting the Blob Containers usually helped. Apparently, there's some state stored in there which isn't always disposed of properly when (re-)adding and removing WebJobs.

Deploying to Azure

The next thing we want to do is deploy our WebJob to Azure. In order for a WebJob to run it needs some executable script that it can use to get going. Many file types are supported, but for us Windows users it makes sense to use an exe, cmd, bat or PowerShell file.

A Console App used to be an exe file, but in .NET Core, it produces a regular DLL file that we need to start manually. So, create a file and name it "run.cmd" and make sure that it's encoded in UTF-8 no BOM (you can check this using something like Notepad++). It just needs a single line of code, which is "dotnet NetCoreWebJob.WebJob.dll". This runs your Console App. Make sure you set the "Copy to Output Directory" of the file to "Copy always".

One last thing, for some reason Azure WebJobs needs all the dependencies of a WebJob, which means all .NET Core packages we used to build it. You can do this by editing the csproj file and adding "<CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>" to the first <PropertyGroup> (underneath "<TargetFramework>").

Before we can deploy our WebJob we need to deploy our web app. Right-click the ASP.NET project and click "Publish...". Simply follow the wizard and Visual Studio will deploy your app for you. You can create a new web app or select an existing one. This step isn't strictly necessary as you can host stand-alone WebJobs, but this should be familiar and it gives you an App Service we can use for our WebJob.

Deploy using Visual Studio

Deploying WebJobs using Visual Studio should be easy as pie. In fact, you probably already know how to do this (don't do it yet though). Right-click your WebJob project and click "Publish...". The following wizard looks a whole lot like the publication of a web app, which we just did. You can pick "Select existing" and pick the Azure web app we just created.

Unfortunately, Microsoft messed up this feature in the worst way possible. Visual Studio will deploy the WebJob with the same name as the project, which is "NetCoreWebJob.WebJob", except the dot is an illegal character in a WebJob name! This messed up my project so bad I had to manually edit it to make my solution working again. Nice one, Microsoft!

So here's what you do. At the start of the wizard, where you pick either a new or existing App Service, click the arrow next to "Publish immediately" and pick "Create Profile" instead. Now you can first change the name of your WebJob in the settings and deploy after that. Make sure you don't select "Remove additional files at destination"  or you'll remove your web app.

Now, browse to the Azure Portal and look up your web app. You'll find "WebJobs" in the menu. You'll see your WebJob, but it's not doing anything. You need to manually run it by selecting it and clicking "Run". The status should update to "Running". You can now check out the logs to see that it actually works. You may see an error about the connection strings, but you can ignore those. If you toggle the output you'll still see a log is written to the console which lets you know it works! If you don't see a log right away try waiting a minute or two and don't forget to manually refresh the output.

[caption id="attachment_10025" align="alignnone" width="867"]WebJobs in Azure WebJobs in Azure[/caption]

Deploy using the Azure Portal

When you add a new WebJob you'll need to fill out some options. You can make up some name, set the type to "Triggered" and the triggers to "Manual". Your alternatives are a "Continuous" WebJob, which just runs and closes (unless you've implemented an endless loop in your application); and a "Scheduled" triggered job, which is basically what we have except we've implemented the schedule ourselves.

The "File upload" needs a bit of explanation. Here you can upload a zip file that contains your WebJob. So head over to Visual Studio and build your solution. Then go to the output folder of your WebJob project, something like "MyProject\bin\[Debug|Release]\netcoreapp2.1", and put everything in that folder into a zip file. Then select it in the "File upload" in your new WebJob.

[caption id="attachment_10035" align="alignnone" width="520"]Add WebJob Add WebJob[/caption]

It will take a few seconds for Azure to create your WebJob so keep refreshing until it pops up. After that, you have to start it manually again and you can check out the logs.

Deploy using VSTS

Ultimately, we want to add our WebJob to our CI/CD pipeline in VSTS. Unfortunately, this functionality doesn't come out of the box. Luckily, it's not very difficult either. If you're not familiar with builds or releases in VSTS check out one of my previous blogs Azure Deployment using Visual Studio Team Services (VSTS) and .NET Core or ARM Templates to Deploy your Resources to Azure.

When you're in the Azure Portal find the "App Service Editor (Preview)" of your App Service. This lets you browse all the files in your App Service. One thing we notice is that your WebJob is located in "App_Data\jobs\triggered\[WebJob name]". And since your WebJob is really just the output of the WebJob project build it's simply a matter of copying your WebJob files to App_Data.

[caption id="attachment_10045" align="alignnone" width="452"]WebJob file location WebJob file location[/caption]

The build

So go to VSTS and create a new build. Select your repository, your branch, and select "ASP.NET Core" as a template. We only need to change two things here. We need to change the existing "Publish" task and add a new ".NET Core" task to publish our WebJob.

Change the name of the existing publish task to "Publish web app", untick the "Publish Web Projects" checkbox, and enter the "Path to project(s)", which is "**/NetCoreWebJob.csproj". Also, untick the "Zip Published Projects" and "Add project name to publish path" checkboxes as they will eventually mess up our release.

After that create a new .NET Core task, set the "Command" to "publish" and change the name of the task to "Publish web job". Again, untick "Publish Web Projects" and set the "Path to project(s)", which is "**/NetCoreWebJob.WebJob.csproj". And once again, don't zip the published projects or add the project name to the publish path. The last step here is the "Arguments" field, which can be copy/pasted from the other publish step, except we're going to add a bit to it: "--configuration $(BuildConfiguration) --output $(build.artifactstagingdirectory)\App_Data\jobs\triggered\WebJobVSTS".

[caption id="attachment_10055" align="alignnone" width="851"]VSTS WebJob build VSTS WebJob build[/caption]

The release

Last, but not least, is the release. Create a new release in VSTS, pick the "Azure App Service deployment" template and fill in the blanks, which is an artifact and your Azure settings in the Environment. Because we're not zipping our build we just need to change one setting. In the "Deploy Azure App Service" task is a "Package or folder" setting, which has a default of "[...]/*.zip" which obviously isn't going to work. Instead, use the browser (the button with "...") and select your drop folder.

Save it, hit new release and pick your latest build. If all goes well you should see your new WebJob in the Azure Portal!

Wrap up

Hopefully, Microsoft will come with a built-in solution for creating, consuming, and deploying WebJobs to Azure in .NET Core soon. Until that time it's not particularly difficult, it's just a matter of knowing what to do.

In this post, we've seen the TimerTrigger, but there's a QueueTrigger, a BlobTrigger, and a FileTrigger as well. Experiment, google, and read the official documentation.

Don't forget to delete your resources.

Happy coding!