A few weeks ago, I wrote a couple of blog posts on how to set up continuous deployment to Azure Web Apps, and how to get Gulp to run as a part of it. I covered how to do it from GitHub using Kudu, and how to do it from VSTS using XAML-based build definitions. However, I never got around to do a post about how to do it using the new scripted build definitions in VSTS. So that is why this post is going to be about!
The application I’ll be working with, is the same on that I have been using in the previous posts. So if you haven’t read them, you might want to go and have a look at them. Or, at least the first part of the first post, which includes the description of the application in use. Without that knowledge, this post might be a bit hard to follow…
If you don’t feel like reading more than you need to, the basics are these. It’s an ASP.NET web application that uses TypeScript and LESS, and Gulp for generating transpiled, bundled and minified files versions of these resources. The files are read from the Styles and Scripts directories, and built to a dist directory using the default” task in Gulp. The source code for the whole project, is placed in a Src directory in the root of the repo…and the application is called DeploymentDemo.
I think that should be enough to figure out most of the workings of the application…if not, read the first post!
Setting up a new build
Ok, so the first step is to set up a new build in our VSTS environment. And to do this, all you need to do, is to log into visualstudio.com, go to your project and click the “Build” tab
Next, click the fat, green plus sign, which gives you a modal window where you can select a template for the build definition you are about to create. However, as I’m not just going to build my application, but also deploy it, I will click on the “Deployment” tab. And since I am going to deploy an Azure Web App, I select the “Azure Website” template and click next.
Note: Yes, Microsoft probably should rename this template, but that doesn’t really matter. It will still do the right thing.
Warning: If you go to the Azure portal and set up CD from there, you will actually get a XAML-based build definition, and not a scripted one. So you have to do it from in here.
Note: VSTS has a preview feature right now, where you split up the build and deployment steps into multiple steps. However, even if this is a good idea, I am just going to keep it simple and do it as a one-step procedure.
On the next screen, you get to select where the source code should come from. In this case, I’ll choose Git, as my solution is stored in a Git-based VSTS project. And after that, I just make sure that the right repo and branch is selected.
Finally, I make sure to check the “Continuous integration…”-checkbox, making sure that the build is run every time I push a change.
That’s it! Just click “Create” to create build definition!
Note: In this window you are also asked what agent queue to use by default. In this example, I’ll leave it on “Hosted”. This will give me a build agent hosted by Microsoft, which is nice. However, this solution can actually be a bit slow at times, and limited, as you only get a certain amount of minutes of free builds. So if you run into any of these problems, you can always opt in to having your own build agent in a VM in Azure. This way you get dedicated resources to do builds. Just keep in mind that the build agent will incur an extra cost.
Once that is done, you get a build definition that looks like this
Or at least you did when I wrote this post…
As you can see, the steps included are:
1. Visual Studio Build – A step that builds a Visual Studio solution
2. Visual Studio Test – A step that runs tests in the solution and makes sure that failing tests fail the build
3. Azure Web App Deployment – A step that publishes the build web application to a Web App in Azure
4. Index Sources & Publish Symbols – A step that creates and published pdb-files
5. Copy and Publish Artifacts – A step that copies build artifacts generated by the previous steps to a specified location
Note: Where is the step that downloads the source from the Git repo? Well, that is actually not its own step. It is part of the definition, and can be found under the “Repository” tab at the top of the screen.
In this case however, I just want to build and deploy my app. I don’t plan on running any tests, or generate pdb:s etc, so I’m just going to remove some of the steps… To be honest, the only steps I want to keep, is step 1 and 3. So it looks like this
Configuring the build step
Ok, now that I have the steps I need, I guess it is time to configure them. There is obviously something wrong with the “Azure Web App Deployment” step considering that it is red and bold… But before I do anything about that, I need to make a change to the “Visual Studio Build” step.
As there will be some npm stuff being run, which generates that awesome, and very deep, folder structure inside of the “node_modules” folder, the “Visual Studio Build” step will unfortunately fail in its current configuration. It defines the solution to build as **/*.sln, which means “any file with an .sln-extension, in any folder”. This causes the build step to walk through _all_ the folders, including the “node_modules” folder, searching for solution files. And since the folder structure is too deep, it seems to fail if left like this. So it needs to be changed to point to the specific solution file to use. In this case, that means setting the Solution setting to Src/DeploymentDemo.sln. Like this
Configuring the deployment step
Ok, so now that the build step is set up, we need to have a look at the deployment part. Unfortunately, this is a bit more complicated than it might seem, and to be honest, than it really needed to be. At first look, it doesn’t look too bad
Ok, so all we need to do is to select the subscription to use, the Web App to deploy to and so on… That shouldn’t be too hard. Unfortunately all that becomes a bit more complicated when you open the “Azure Subscription” drop-down and realize that it is empty…
The first thing you need to do is to give VSTS access to your Azure account, which means adding a “Service Endpoint”. This is done by clicking the Manage link to the right of the drop-down, which opens a new tab where you can configure “Service Endpoints”.
The first thing to do is to click the fat, green plus sign and select Azure in the drop-down. This opens a new modal like this
There are 3 different ways to add a new connection, Credentials, Certificate Based and Service Principle Authentication. In this case, I’ll switch over to Certificate Based.
Note: If you want to use Service Principle Authentication you can find more information here
First, the connection needs a name. It can be whatever you want. It is just a name.
Next, you need to provide a bunch of information about your subscription, which is available in the publish settings file for your subscription. The easiest way to get hold of this file, is to hover over the tooltip icon, and then click the link called publish settings file included in the tool tip pop-up.
This brings you to a page where you can select what directory you want to download the publish settings for. So just select the correct directory, click “Submit”, and save the generated file to somewhere on your machine. Once that is done, you can close down the new tab and return to the “Add New Azure Connection” modal.
To get hold of the information you need, just open the newly downloaded file in a text editor. It will look similar to this
As you can see, there are a few bits of information in here. And it can be MUCH bigger than this if you have many subscriptions in the directory your have chosen. So remember to locate the correct subscription if you have more than one.
The parts that are interesting would be the attribute called Id, which needs to be inserted in the Subscription Id field in the modal, the attribute called Name, which should be inserted in Subscription Name, and finally the attribute called ManagementCertificate, which goes in the Management Certificate textbox. Like this
Once you click OK, the information will be verified, and if everything is ok, the page will reload, and you will have a new service endpoint to play with. Once that is done, you can close down the tab, and return to the build configuration set up.
The first thing you need to do here, is to click the “refresh” button to the right of the drop-down to get the new endpoint to show up. Next, you select the newly created endpoint in the drop-down.
After that, you would assume that the Web App Name drop-down would be populated with all the available web apps in your subscription. Unfortunately, this is not the case for some reason. So instead, you have to manually insert the name of the Web App you want to deploy to.
Note: You have two options when selecting the name of the Web App. Either, you choose the name of a Web App that you have already provisioned through the Azure portal, or you choose a new name, and if that name is available, the deployment script will create a new Web App for you with that name on the fly.
Next, select the correct region to deploy to, as well as any specific slot you might be deploying to. If you are deploying to the default slot, just leave the “slot” textbox empty.
The Web Deploy Package box is already populated with the value $(build.stagingDirectory)\**\*.zip which works fine for this. If you have more complicated builds, or your application contains other zips that will be output by the build, you might have to change this.
Once that is done, all you have to do is click the Save button in the top left corner, give the build definition a name, and you are done with the configuration.
Finally, click the Queue build… button to queue a new build, and in the resulting modal, just click OK. This will queue a new build, and give you a screen like this while you wait for an agent to become available
Note: Yes, I have had a failing build before I took this “screen shot”. Yours might look a little bit less red…
And as soon as there is an agent available for you, the screen will change into something like this
where you can follow along with what is happening in the build. And finally, you should be seeing something like this
At least if everything goes according to plan
Adding Gulp to the build
So far, we have managed to configure a build and deployment of our solution. However, we are still not including the Gulp task that is responsible for generating the required client-side resources. So that needs to be sorted out.
The first thing we need to do is to run
To do this, click the fat, green Add build step… button at the top of the configuration
and in the resulting modal, select Package in the left hand menu, and then add an npm build step
Next, close the modal and drag the new build step to the top of the list of steps.
By default, the command to run is set to install, which is what we need. However, we need it to run in a different directory than the root of the repository. So in the settings for the npm build step, expand the Advanced area, and update the Working Directory to say Src/DeploymentDemo.
Ok, so now npm will install all the required npm packages for us before the application is built.
Next, we need to run
To do this, add a new build step of the type Command Line from the Utility section, and drag it so that it is right after the npm step. The configuration we need for this to work is the following
Tool should be $(Build.SourcesDirectory)\Src\DeploymentDemo\node_modules\.bin\bower.cmd, the arguments should be install, and the working folder should be Src/DeploymentDemo
This will execute the bower command file, which is the same as running Bower in the command line, passing in the argument install, which will install the required bower packages. And setting the working directory will make sure it finds the bower.json file and installs the packages in the correct folder.
Now that the Bower components have been installed, or at least been configured to be installed, we can run Gulp. To do this, just add a new Gulp build step, which can be found under the Build section. And then make sure that you put it right after the Command Line step.
As our gulpfile.js isn’t in the root of the repo, the Gulp File Path needs to be changed to Src/DeploymentDemo/gulpfile.js, and the working directory once again has to be set to Src/DeploymentDemo
As I’m using the default task in this case, I don’t need to set the Gulp Task(s) to get it to run the right task.
Finally, I want to remove any left over files from the build agent as these can cause potential problems. They really shouldn’t, at least not if you are running the hosted agent, but I have run in to some weird stuff when running on my own agent, so I try to always clean up after the build. So to do this, I will run the batch file called delete_folder.bat in the Tools directory of my repo. This will use RoboCopy to safely remove deep folder structures, like the node_modules and bower_components folders.
To do this, I add two new build step to the end of the definition. Both of them of the type Batch Script from the Utility section of the Add Build Step modal.
Both of them need to have their Path set to Tools/delete_folder.bat, their Working Folder set to Src/DeploymentDemo, and their Always run checkbox checked. However, the first step needs to have the Arguments set to node_modules and the second one have it set to bower_components
This will make sure that the bower_components and node_modules folders are removed after each build.
Finally save the build configuration and you should be done! It should look something like this
However, there is still one problem. Gulp will generate new files for us, as requested, but it won’t be added to the deployment unfortunately. To solve this, we need to tell MSDeploy that we want to have those files added to the deployment. To do this, a wpp.targets-file is added to the root of the project, and checked into source control. The file is in this case called DeploymentDemo.wpp.targets and looks like this
<?xml version="1.0" encoding="utf-8" ?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Target Name="AddGulpFiles" BeforeTargets="CopyAllFilesToSingleFolderForPackage;CopyAllFilesToSingleFolderForMsdeploy">
<Message Text="Adding gulp-generated files to deploy" Importance="high"/>
<CustomFilesToInclude Include=".\dist\**\*.*" />
It basically tells the system that any files in the dist folder should be added to the deployment.
Note: You can read more about wpp.targets-files and how/why they work here: https://chris.59north.com/post/Integrating-a-front-end-build-pipeline-in-ASPNET-builds
That’s it! Queuing a new build, or pushing a new commit should cause the build to run, and a nice new website should be deployed to the configured location, including the resources generated by Gulp. Unfortunately, due to the npm and Bower work, the build can actually be a bit slow-ish. But it works!