andy-barnes.co.uk

Home

Adding a What-If to an ARM Azure DevOps Pipeline

1 September 20212 September 2021
by andybadmin

When deploying ARM templates via Azure PowerShell/CLI or Azure DevOps pipelines, it is extremely useful to know what actions will be performed.
Sure creating a storage account is simple to understand the outcome, but in a complicated existing environment knowing what resources will be added, modified or removed is very important to validate.

Microsoft added a new What-If option to the Azure PowerShell and Azure CLI modules, which outputs exactly what changes it will make, so you can confirm the actions before running the deployment.
This is very similar to a Terraform “Plan” which will validate which resources will be added, modified or destroyed and helps identify any obvious configuration issues or dependencies before a Terraform “Apply” aka deployment.

For information on What-If can be found here on MS Docs: ARM template deployment what-if operation – https://docs.microsoft.com/en-us/azure/azure-resource-manager/templates/deploy-what-if?tabs=azure-powershell

Its great being able to run a What-If on the command line, but what if you’re using Azure DevOps pipelines to deploy ARM Templates? Can you make use of this?

Well the good news is a What-If task can be added to your pipeline and its also simple to do so.
My preference is to add a What-If task in two places; the build pipeline to validate the ARM template is going to do what I expect; and also the release pipeline just as a simple additional bit of pipeline logging so I can see what the ARM template as going to do right before the deploy task runs.

Below I will detail how to add a What-If task to both an existing Build Pipeline (Classic) and a Release Pipeline.

Build Pipeline (Classic)

Navigate to your existing Build Pipeline and select “Edit”.

Add a task to the Agent job by clicking “+”.

Select “Azure PowerShell” and click “Add”.

Give the task a name of “ARM Template Deployment What-If”.

Select your Subscription.

Select “Inline Script” and the Script type.

For Azure PowerShell version, select “Latest installed version”.

Enter the following inline script amending as appropriate for your template file, parameter file and resource group:

New-AzResourceGroupDeployment -TemplateFile ARMTemplates/azuredeploy.json -TemplateParameterFile ARMTemplates/azuredeploy.parameters.json -ResourceGroupName "RG-UKS-T-ARM-LAB-01" -WhatIf

Click “Save & queue” to run the Build Pipeline.

Enter a comment if desired then click “Save & run”.

The job will be queued onto an agent.

As the tasks in the Build Pipeline run though, it will get to the What-If Azure PowerShell task and output exactly what Create, Modify and Delete actions will be performed based on the template, parameters and resource group provided.

In this case it will be creating a new storage account with the properties listed.

Another example could be modifying a storage account from LRS to GRS, the SKU of the storage account would show as a modification.

Release Pipeline

Navigate to your existing Release Pipeline and select “Edit”.

Select your desired stage which performs an ARM Template deployment.

Add a task to the Agent job by clicking “+”.

Select “Azure PowerShell” and click “Add”.

Give the task a name of “ARM Template Deployment What-If”.

Select your Subscription.

Select “Inline Script” and the Script type.

For Azure PowerShell version, select “Latest installed version”.

Enter the following inline script amending as appropriate for your template file, parameter file and resource group:

New-AzResourceGroupDeployment -TemplateFile "$(System.DefaultWorkingDirectory)/_Build Pipeline ARM Template Storage Account/drop/azuredeploy.json" -TemplateParameterFile "$(System.DefaultWorkingDirectory)/_Build Pipeline ARM Template Storage Account/drop/azuredeploy.parameters.json" -ResourceGroupName "RG-UKS-T-ARM-LAB-01" -storageName $(storageName) -environment $(environment) -WhatIf

Click Save.

Click “Create Release” and “Create” again.

Depending on your Pipeline trigger settings, it will either automatically trigger a run after the release (as in my case) or you will need to trigger it manually.

You will see the new Release and the stage beginning to run.

The job will be queued on an agent and the tasks run.

As mentioned earlier, I like to add a What-If task to a Release pipeline simply to add further logging of exactly what is being deployed in the next task because it can make troubleshooting easier.

If you view the log for the “ARM Template Deployment What-If” task, the output of what will be added, modified or deleted is displayed. In this case it is simply creating a new storage account.

Hopefully this helps you utilise the power and usefulness of What-If in Azure DevOps pipelines.

Azure

Create an Azure DevOps Release Pipeline for ARM Template…

30 August 20211 September 2021
by andybadmin

Azure DevOps provides pipelines for both Continuous Integration (CI) aka “Build” and Continuous Delivery (CD) aka “Release”.

In this post I will create a Release Pipeline to deploy an ARM Template using an Artifact produced in a Build Pipeline we created previously here: Create an Azure DevOps Build Pipeline (Classic) for ARM Template Deployment.

Artifacts

The end result of our Build pipeline, was an Artifact containing our validated ARM Template.

This Artifact can be viewed by clicking on the “1 published” link on the Build pipeline run summary page.

This will take you to the Artifact within a drop folder containing our azuredeploy.json and azuredeploy.parameters.json files which create a storage account.

Release Pipeline

We are now going to create a Release pipeline to perform an ARM template deployment and create our storage account.

Navigate to Pipelines –> Releases.

Click on “+ New” and select “+ New release pipeline”.

Choose “Empty pipeline” when provided with the template options below.

Now we have an empty pipeline, we first need to add our Artifact.

Click on “Add an artifact”.

Select the following:

Source type: Build
Project: (your project name)
Source: (your build pipeline name)
Default version: Latest (or latest from a branch such as main/master)

Click Add to add the Artifact to the pipeline.

Click on the lightning icon on the Artifact. Here you can configure triggers so that the release pipeline runs after a build pipeline us run. For now we will leave this disabled and trigger the release pipeline manually.

Stages

Click on the lightning icon on the Stage 1. Here you can configure the trigger to run as soon as a new release is created. We will ensure this is set to “After release”.

Now we need to configure our stage.

Click on 1 job, 0 task.

Give the stage an appropriate name, such as “Create Storage Account”.

Click on “Agent Job”, review the settings and change Agent specification to “windows-2019” or your desired ADO agent OS.

All other settings can be left for now, but here you can choose to use self-hosted agents for this stage if you have them and parallel job execution.

Tasks

Next we add a Task to the Job.

Click the “+” and add “ARM Template deployment”.

Enter a name such as “ARM Template deployment: Storage Account”.

Configure the following settings as required:

Deployment Scope: Resource Group
Azure Resource Manager connection: <this should have been setup when the project was created>
Subscription: <your subscription to deploy to>
Action: Create or update resource group
Resource Group: e.g. RG-UKS-T-ARM-LAB-01
Location: UK South
Template Location: Linked Artifact
Template: $(System.DefaultWorkingDirectory)/_Build Pipeline ARM Template Storage Account/drop/azuredeploy.json
Template Parameters: $(System.DefaultWorkingDirectory)/_Build Pipeline ARM Template Storage Account/drop/azuredeploy.parameters.json
Deployment Mode: Incremental

Give your pipeline a name, such as “Release Pipeline Storage Account”.

Click Save.

Release

Click “Create Release”.

Enter a release description if desired and click “Create”.

The release is created (in my case its the second release) and it will begin to run automatically because we set the stage to trigger “after release” earlier in the pipeline configuration.

Click on “Release-1” or the number of your release and view the pipeline progress.

It will first queue for an agent to run the pipeline job.

The agent will then run the tasks, such as downloading the Artifact and the ARM Template deployment.

This will only take a few seconds for our storage account creation to complete, at which point it will show as “Succeeded”.

Looking back at the releases for the pipeline, it also confirms from this view that the “Create Storage Account” Stage in the release was successful.

Finally we verify the Storage Account was created in our Resource Group.

To summarise what we have accomplished…
We created a release pipeline which takes an Artifact from a build pipeline. It then uses the ARM Template and parameter file within the Artifact to deploy a storage account.
Whenever a new release is created when changing or adding stages or job tasks within the release pipeline, a new run will automatically trigger after release.

Azure

Create an Azure DevOps Build Pipeline (Classic) for ARM…

19 August 20211 September 2021
by andybadmin

Azure DevOps provides pipelines for both Continuous Integration (CI) and Continuous Delivery (CD). It also provides Git based Repos to store source files (in our case ARM templates) and version control them.

In this post I will create a Build Pipeline (also known called as a “Classic” build pipeline) to validate the ARM template and create an Artifact.
Then in a follow up post I will create a Release Pipeline (CD) to deploy the ARM template based on the Artifact created in this Build Pipeline (CI).

There are lots of ways of working with Pipelines in Azure DevOps (ADO). This is one way of doing it.
Another way is skip the build pipeline all together and go straight to release without creating an Artifact, or use the newer YAML based pipeline which allows your pipeline itself to be stored as code rather than using a GUI (aka Classic editor).

Prerequisites:

An Azure subscription
An Azure DevOps organisation, project and service connection to the subscription
An ARM template in the Repo (such as this one to create a storage account https://github.com/Azure/azure-quickstart-templates/tree/master/quickstarts/microsoft.storage/storage-account-create)

Azure DevOps

Select an existing project or create a new one (e.g. “ARM Lab”).

Azure Repos

Navigate to the Repo within the project.

Create a folder called “ARMTemplates”.

Upload your ARM template (e.g. azuredeploy.json) and parameter file (e.g. azuredeploy.parameters.json) to the Repo.

If you don’t already have an ARM template you can use this one from the Azure quickstart GitHub repo: https://github.com/Azure/azure-quickstart-templates/tree/master/quickstarts/microsoft.storage/storage-account-create

Azure Pipelines (Build Pipeline)

Navigate to Pipelines –> Pipelines.

This is where a Build Pipeline (Classic) is created, or a new YAML pipeline. It used to be labelled as Build Pipeline.

Click “New pipeline”.

Select “Use the classic editor”. This will create a build pipeline using the GUI.

If you selected anything else you will have ended up creating a YAML pipeline which is not the scope of this post. Please go back and select “Use the classic editor”.

Now we have to select where our code is located, choose “Azure Repos Git”.

Select the appropriate project, repository and default branch (e.g. main) and click Continue.

Choose “Empty pipeline” when provided with the template options below.

Give your pipeline a name, such as “Build Pipeline ARM Template Storage Account”, that seems to explain it well enough, but by all means call yours “Test Pipeline 1” if you wish 🙂

Leave the “Azure Pipelines” agent pool selected which will use the Microsoft hosted ADO agents.

For Agent Specification choose “windows-2019”

Note: There is a free tier for Microsoft hosted agents which allows 1 free parallel job. You may need to request access and details can be found here: https://docs.microsoft.com/en-us/azure/devops/pipelines/licensing/concurrent-jobs?view=azure-devops&tabs=ms-hosted

Select “Get Sources” under the Pipeline, ensure the settings are correct from what you specified earlier.

Click on “Agent job 1”, there is nothing to configure where specifically in our case, but here you can how parallel jobs run and timeouts.

Click on “+” next to “Agent job1”, this where we add tasks.

A number of different types of tasks will be listed to choose from. These cover everything to deploying and testing code, specific service deployments in Azure, or as in our case ARM template deployment. You can even search a market place of tasks and deploy to other public and private clouds such as AWS and VMware.

Select “ARM template deployment” and click “Add”.

Change the task name to “ARM Template Validation”. Because in our case this is a build pipeline and we are only going to check the template is valid. We will deploy it in a Release Pipeline later.

Configure the following settings as required:

Deployment Scope: Resource Group
Azure Resource Manager connection: <this should have been setup when the project was created>
Subscription: <your subscription to deploy to>
Action: Create or update resource group
Resource Group: e.g. RG-UKS-T-ARM-LAB-01
Location: UK South
Template Location: Linked Artifact
Template: ARMTemplates/azuredeploy.json
Template Parameters: ARMTemplates/azuredeploy.parameters.json
Deployment Mode: Validation Only

Click on “+” next to “Agent job1” again to add another task.

Select “Publish build artifacts” this time and click “Add”.

Configure the following:

Path to publish: ARMTemplates
Artifact name: drop
Artifact publish location: Azure Pipelines

Navigate to the sub tab of “Triggers”.

Tick “Enable continuous integration”.
Add the “main” branch to the Branch filter.
Add “ARMTemplates” to the Path filer.

This will trigger our build pipeline to run every time where is a commit to the ARMTemplates folder in the main branch.
This is not enabled by default, but without this you are not working in a true Continuous Integration (CI) manner, and would have to manually run a build each time.
Each time the ARM template is amended, it will be automatically validated.

All that is left is to “Save & queue” our Build Pipeline using the button at the top.

Validate the agent settings and hit “Save and run”.

Select “Agent job 1” to see the status of the pipeline run.

The Status will reflect “Success” once all the tasks have completed without any issues.

You can review the logs of each task as it runs in realtime, including the code being checked out from our Azure Repo and the main branch using Git.

Next is our ARM template validation task which is successful.

Following the success of the validation (and only if that task is successful), an Artifact is created which will be used by a Release Pipeline later.

This Artifact can be viewed by clicking on the “1 published” link on the pipeline run summary page.

This will take you to the Artifact with a drop folder containing our azuredeploy.json and azuredeploy.parameters.json files.

To summarise what we have just done… We added our ARM template to an Azure Repo, created an ADO Build Pipeline (Classic), created a task to validate the ARM template, created a task to publish an Artifact ready to be consumed by a Release Pipeline.
Any commits to the ARMTemplate folder within the Repo will automatically trigger the Build Pipeline.

See here for the next step which is: Create an Azure DevOps Release Pipeline for ARM Template Deployment

Azure

Create an Azure Bastion

7 August 20212 September 2021
by andybadmin

Azure Bastion is a PaaS service which allows you to securely connect to Virtual Machines using the Azure Portal using HTTPS without having to expose RDP or SSH to the internet via a public IP.

This removes the risk of having a jump box exposed to the internet which is insecure. Azure Bastion also ensures anyone connecting is already authenticated via the Azure portal, before being prompted for server login credentials.
It is simple to deploy a bastion host and there aren’t any additional extensions to install on the VM or within your browser for this to work.

“Azure Bastion is a fully managed service that provides more secure and seamless Remote Desktop Protocol (RDP) and Secure Shell Protocol (SSH) access to virtual machines (VMs) without any exposure through public IP addresses. Provision the service directly in your local or peered virtual network to get support for all the VMs within it.”
https://azure.microsoft.com/en-gb/services/azure-bastion/

Azure Bastion requires its own subnet which must be named “AzureBastionSubnet” and this subnet should be at least a /27 in size.

You can connect to any VM within the same vNet as Azure Bastion and any VM within any vNet peered directly with the vNet Bastion is deployed within, however it cannot traverse multiple vNet peerings.

There are two SKUs for Azure Bastion:

Basic
- Basic deploys 2 Bastion hosts.
Standard
- Standard allows a configurable number of Bastion hosts between 2 and 50 enabling more sessions if required.

Below is a diagram of what we are going to create:

1 vNet with 2 Subnets (AzureBastionSubnet and snet-uks-t-mgmt-vms)
1 Windows VM connected to snet-uks-t-mgmt-vms
1 Bastion Standard connected to AzureBastionSubnet

Create the Virtual Network

Create a Virtual Network

select your Subscription
select or create a Resource Group
enter a Virtual Network Name
select a Region

2. Enter an Address Space for the Virtual Network e.g. 10.20.0.0/24.

3. Create 2 Subnets:

snet-uks-t-mgmt-vms with an address range of 10.20.0.0/26
AzureBastionSubnet with an address range of 10.20.0.64/27

Note: Azure Bastion requires its own subnet which must be named “AzureBastionSubnet” and this subnet should be at least a /27 in size.

Create the Virtual Machine

4. Create a Virtual Machine and connect it to the subnet “snet-uks-t-mgmt-vms”.

OS: Windows Server 2019 Datacenter
Size: B2s
Disk: Standard HDD
Subnet: snet-uks-t-mgmt-vms
Public IP: No public IP

Create the Bastion

5. Search for “Bastion” and click Create Bastion.

6. Create a Bastion with the following settings:

select your Subscription
select or create a Resource Group
Bastion Name: bst-uks-t-mgmt-01
Region: UK South
Tier: Standard
Instance Count: 2
Virtual Network: vnet-uks-t-mgmt
Subnet: AzureBastioSubnet
Public IP: pip-bst-uks-t-mgmt-01

7. Review the settings and click Create.

8. The Bastion will take a few minutes to deploy. Once complete you will see the Provisioning State “Succeeded” as below.

Connect to a VM via the Bastion

9. To connect to a Virtual Machine via the new Bastion, navigate to the VM created earlier within the same Virtual Network, select Connect and Bastion from the sub menu.