12 Things you Should Know when Implementing Azure DevOps in your Organization

Azure DevOps is a really fantastic part of any DevOps tool chain. But when you’re first starting out with it in an organization, there are a few things you should know that will make it even better… and avoid making some doing some things you’ll later regret. These tips are most important if you’re implementing it across multiple teams or in a medium to large organization. Even if you’re implementing it in a small start-up, most of these tips will still help.

These tips are all based on my experience with implementing and using Azure DevOps, Visual Studio Online (VSO) and Visual Studio Team Services (VSTS). These are all things I wish I’d known earlier as they would have saved me time, made my life easier or kept me more secure. They are also just my opinion, so I encourage you to investigate further and decide what is best for you in your situation/environment.

This is by no means an exhaustive list either and they are in no particular order.

So, let’s get into it:

1. Projects: less is better.

Less projects are better

Most things (work items, repositories, pipelines etc.) in Azure DevOps are organized into containers called Projects. It is tempting to try to break your work into lots of small projects (e.g. one for each library, or one per team, or one per department). This results in a lot of management overhead trying to keep everything organized and adds little value in most cases (there are exceptions). Implementing a project per team or a project per software component is usually wrong.

Recommendation: The less projects you have the better. Use Area Paths (covered next) to organize work in your project.

Documentation Reference: When to add another project

2. Area Paths: Organize work.

Organizing Area Paths

Area Paths allow you to divide the work items and test plans into a hierarchy to make them easier to manage. Teams can be assigned to one or more area paths.

Area’s are easily moved around, so they are much better suited to arranging your work by software component/product and organizational hierarchies.

Recommendation: For your project, set up Area Paths and assign teams to them.

Documentation Reference: Define area paths for your project

3. Identity: Integrate with Azure AD.

Connect AAD.

If you are using Azure AD as your primary identity source for your organization, then you should connect your Azure DevOps organization to Azure AD. This will allow your Azure AD identities to be used within Azure DevOps.

If you aren’t using Azure AD, but have Active Directory, consider setting up hybrid identity with Azure AD.

You should manage access to your Azure DevOps organization and to projects and other resources (e.g. Service Connections) using Azure AD Groups. I’d also strongly recommend reading the documentation on security groups and permissions for Azure DevOps as there are a lot of nuance to these and they deserve an entire post on their own.

Recommendation: Use Azure AD as the identity source for Azure DevOps. Create and manage users and security groups within Azure AD.

Documentation Reference: Connect organization to Azure Active Directory, Access with Active Directory Groups.

4. Git or TFVC?

Importing a TFVC repository as Git

This might be controversial, but it shouldn’t be. Unless you have a legacy TFVC repository that you need to keep around for historic/support reasons or some tools that only support TFVC that you can’t live without (or can’t replace) then you should be using Git as your version control system.

If you do have legacy TFVC repositories that you need to bring over, consider importing them as Git repositories.

Recommendation: Use Git. Make sure all your teams know how to use Git well.

Documentation Reference: When to add another project

5. Create a Sandbox Project.

Create a Sandbox Project

You and your teams will often need a place to experiment and learn about Azure DevOps safely. A sandbox project is a great place to do this. You can create a sandbox project and give teams higher levels of permissions over it project to allow them to experiment with different settings (for example try out an alternate area path structure)

Don’t confuse a sandbox project with a project for building proof-of-concepts/experimental code: you should not use a Sandbox project for creating anything that could end up in production or anything that has any value. Content in sandbox projects often accidentally get deleted.

Recommendation: Create a Sandbox Project. Assign an image and description for the project to make it clear that it is a Sandbox and what it can be used for.

Documentation Reference: When to add another project

6. Install extensions… wisely

The Azure DevOps Extensions Marketplace

The Azure DevOps marketplace is filled with many great extensions that really enhance the value of Azure DevOps. It is well worth browsing through the extensions created by both Microsoft, Microsoft DevLabs and the hundreds of 3rd party ones to really experience the full power of Azure DevOps.

You should set up a formal process around validating, on-boarding and off-boarding extensions from your organization. It is all too easy to end up with “extension sprawl” that results in a management nightmare, especially if you have strict security or governance practices (you might be familiar with this if you’ve ever managed Jenkins within a large organization).

It is also tempting to install pipeline extensions for any minor task that you might want to execute in a CI/CD pipeline. But you should consider if the governance & management of a task is worth the time that might be saved using it, especially when a short Bash or PowerShell task might do just as well.

Recommendation: Install important extensions from marketplace. Formalize a process for validating, on-boarding and off-boarding extensions.

Documentation Reference: Azure DevOps marketplace, Install Azure DevOps Extension

7. Use Multi-stage YAML Pipelines.

Do NOT use “Classic Editor” and create pipelines without YAML

Early on the evolution of Azure DevOps pipelines, all pipelines had to created using a visual designer. The structure of this visually designed pipeline was not stored in code, rather it was stored separately without proper version control. This is no longer the case.

You should always create new build pipelines using YAML and store them in your repository with your source code (pipeline as code). You can still use the assistant to help you design your YAML:

Click Show Assistant to edit your YAML.

The exception is release pipelines which don’t support YAML and being stored in version control.

Recommendation: Create all pipelines a multi-stage YAML pipelines.

Documentation Reference: Define pipelines using YAML syntax

9. Release Pipelines… in code?

The CTO drought and the broken pipeline | by Dmitri Grabov | HackerNoon.com  | Medium
Release Pipelines are awesome, but are they worth missing out on pipeline as code?

Release pipelines don’t support YAML. However, in many cases you don’t need release pipelines. Instead, you can use your multi-stage YAML build pipeline to release your software as well by adding a deployment job. This also aligns much more closely to GitHub, where there is no concept of a Release Pipeline and would make moving to GitHub Actions much easier should you want to.

As of writing this post, there are two key feature that are missing from YAML build pipelines: Gates and Deployment Group jobs. Also, the release pipeline visualization and dashboard widgets are quite useful, so you may prefer these over the build pipeline visualization. But in my opinion the visualization is not worth losing version control over your pipeline.

Recommendation: Use multi-stage YAML pipeline deployments if you don’t need Gates or Deployment Group Jobs. Use conditions to determine if a deployment job should be executed. Use approvals and checks on the environment to control deployment.

Documentation Reference: Deployment Job, Conditions, Approvals, Environments.

10. Deployment Group Agents?

Add a machine to a Deployment Group.

If the applications you are building and releasing need to be deployed to a physical or virtual machine (e.g. not to a Kubernetes cluster or managed service) that is not accessible by an Azure DevOps Hosted agent, then you can use a Deployment Group agent.

This is just the Azure DevOps Hosted agent installed onto the machine and registered with Azure DevOps as a Deployment Group agent in a Deployment Group. Deployment Group agents only require outbound connectivity to Azure DevOps services.

This is a good solution if you’re deploy to machines on-premises or on machines where inbound internet connectivity is blocked, but outbound internet is allowed.

Recommendation: If you have to deploy your application to a machine that is not accessible from Azure DevOps Microsoft Hosted Agents.

Documentation Reference: Deployment Group agent, Deployment Group

11. Automate. Automate.

Get a list of Azure DevOps projects using Azure DevOps CLI

Just like most other Microsoft tools, you can automate them from the command line using either PowerShell, CMD or Bash (or even REST API). If you have to perform repetitive tasks in Azure DevOps, you might want to consider automating these processes.

This is also a good way to control certain processes and practices, such as creating Service Connections from code in a repository, or rolling secrets in a Library Variable Group.

You can also use these tools to interact with Azure DevOps from within Azure DevOps pipelines, leading to some interesting techniques such as release orchestrations (beyond the scope of this doc).

Recommendation: Use Azure DevOps CLI or the VSTeams PowerShell module (created by Donovan Brown) to automate Azure DevOps. Alternatively, use Azure DevOps REST API.

Documentation Reference: Azure DevOps CLI, VSTeam PowerShell module, Azure DevOps REST API.

12. Get Practical Experience.

The best way to learn Azure DevOps is to get hands-on practical experience. Azure DevOps Labs provides free hands-on labs environments (via your own DevOps organization) and covers practically everything you could ever want to know. The Azure DevOps content on Microsoft Learn also has detailed walk throughs of the product and processes.

Making sure everyone in your organization has the skills/knowledge to work with Azure DevOps will help them be more successful and happy.

Recommendation: Do some of the hands-on labs and complete some Microsoft Learn learning pathways.

Documentation Reference: Azure DevOps Labs, Azure DevOps content on Microsoft Learn

Wrapping Up

There are definitely lots more recommendations and considerations I could suggest, especially security and DevOps best-practices but to keep this (reasonably) short, I’ll leave them for another post.

I hope you find this useful and it helps you avoid some of my mistakes.

Deploy Sonarqube to Azure App Service Linux Containers using an Azure DevOps Pipeline

Sonarqube is a web application that development teams typically use during the application development process to continuous validate the quality of the code.

This post is not specifically about Sonarqube and how it works. It is intended to show Developers & IT Pros how to deploy a service to Azure using contemporary infrastructure as code and DevOps patterns.

The Implementation

A Sonarqube installation is made up of a web application front end backed by database.

ss_sonarqube_architecture

Sonarqube supports many different types of databases, but I chose to use Azure SQL Database. I decided to use Azure SQL Database for the following reasons:

    1. It is a managed service, so I don’t have to worry about patching, securing and looking after SQL servers.
    2. I can scale the database performance up and down easily with code. This allows me to balance my performance requirements with the cost to run the server or even dial performance right back at times when the service is not being used.
    3. I can make use of the new Azure SQL Database serverless (spoiler alert: there are still SQL servers). This allows the SQL Database to be paused when not being accessed by the Sonarqube front end. It can be used to further reduce costs running Sonarqube by allowing me to delete the front end every night and only pay for the storage costs when developers aren’t developing code.ss_sonarqube_sql_server_serverless

For the front end web application I decided to use the Azure Web App for Containers running a Linux container using the official Sonarqube Docker image. Because the Sonarqube web application is stateless it is a great target for being able to be delete and recreate from code. The benefits to using Azure Web App for Containers are:

  1. Azure Web App for Containers is a managed service, so again, no patching, securing or taking care of servers.
  2. I can scale the performance up and down and in and out from within my pipeline. This allows me to quickly and easily tune my performance/cost, even on a schedule.
  3. I can delete and rebuild my front end web application by running the pipeline in under 3 minutes. So I can completely delete my front end and save money when it is not in use (e.g. when teams aren’t developing in the middle of the night).

Architectural Considerations

The Sonarqube web application, as it has been architected, is accessible from the public internet. This might not meet your security requirements, so you might wish to change the architecture in the following ways:

  1. Putting an Azure Application Gateway (a layer 7 router) in front of the service.
  2. Isolate the service in a Azure Virtual Network from the internet and make it only accessible to your development services. This may also require Azure ExpressRoute or other VPN technologies to be used.
  3. We are using the SQL Server administrator account for the Sonarqube front end to connect to the backend. This is not advised for a production service – instead, a user account specifically for the use of Sonarqube should be created and the password stored in an Azure Key Vault.

These architectural changes are beyond the scope of this document though as I wanted to keep the services simple. But the pattern defined in this post will work equally well with these architectures.

Techniques

Before we get into the good stuff, it is important to understand why I chose to orchestrate the deployment of these services using an Azure Pipeline.

I could have quite easily built the infrastructure manually straight into the Azure Portal or using some Azure PowerShell automation or the Azure CLI, so why do it this way?

There are a number of reasons that I’ll list below, but this is the most mature way to deploy applications and services.

ss_sonarqube_journey_of_an_Azure_professional

  1. I wanted to define my services using infrastructure as code using an Azure Resource Manager template.
  2. I wanted the infrastructure as code under version control using Azure Repos. I could have easily used GitHub here or one of a number of other Git repositories, but I’m using Azure Repos for simplicity.
  3. I wanted to be able to orchestrate the deployment of the service using a CI/CD pipeline using Azure Pipelines so that the process was secure, repeatable and auditable. I also wanted to parameterize my pipeline so that I could configure the parameters of the service (such as size of the resources and web site name) outside of version control. This would also allow me to scale the services by tweaking the parameters and simply redeploying.
  4. I wanted to use a YAML multi-stage pipeline so that the pipeline definition was stored in version control (a.k.a. pipeline as code). This also enabled me to break the process of deployment into two stages:
    • Build – publish a copy of the Azure Resource Manager templates as an artifact.
    • Deploy to Dev – deploy the resources to Azure using the artifact produced in the build.

Note: I’ve made my version of all these components public, so you can see how everything is built. You can find my Azure DevOps repository here and the Azure Pipeline definition here.

Step 1 – Create a project in Azure DevOps

First up we need to have an Azure DevOps organization. You can sign up for a completely free one that will everything you need by going here and clicking start free. I’m going to assume you have your DevOps organization all set up.

  1. In your browser, log in to your Azure DevOps organization.
  2. Click + Create project to create a new project.
  3. Enter a Project Name and optionally a Description.
  4. Select Public if you want to allow anyone to view your project (they can’t contribute or change it). Otherwise leave it as Private to make it only visible to you.
  5. Click Create.

ss_sonarqube_createproject

You’ve now got an Azure Repo (version control) as well as a place to create Azure Pipelines as well as a whole lot of other tools, such as Azure Boards, that we’re not going to be using for this project.

Step 2 – Add ARM Template Files to the Repo

Next, we need to initialize our repository and then add the Azure Resource Manager (ARM) template files and the Azure Pipeline definition (YAML) file. We’re going to be adding all the files to the repository directly in the browser, but if you’re comfortable using Git, then I’d suggest using that.

  1. Select Repos > Files from the nav bar.
  2. Make sure Add a README is ticked and click Initialize.ss_sonarqube_initializerepo
  3. Click the ellipsis (…) next to the repo name and select Create a new folder.
  4. Set the Folder name to infrastructure. The name matters because the pipeline definition expects to find the ARM template files in that folder.
  5. Enter a checkin comment of “Added infrastructure folder”.
  6. Click Create.ss_sonarqube_createinfrastructurefolder
  7. Once the folder has been created, we need to add two files to it:
    • sonarqube.json – The ARM template representing the infrastructure to deploy.
    • sonarqube.parameters.json – The ARM template default parameters.
  8. Click here to download a copy of the sonarqube.json. You can see the content of this file here.
  9. Click here to download a copy of the sonarqube.parameters.json. You can see the content of this file here.
  10. Click the ellipsis (…) next to the infrastructure folder and select Upload file(s).
  11. Click the Browse button and select the sonarqube.json and sonarqube.parameters.json files you downloaded.
  12. Set the Comment to something like “Added ARM template”.
  13. Ensure Branch name is set to master (it should be if you’re following along).
  14. Click Commit.ss_sonarqube_uploadarmtemplate

We’ve now got the ARM Template in the repository and under version control. So we can track any changes to them.

Note: When we created the infrastructure folder through the Azure DevOps portal a file called _PlaceHolderFile.md was automatically created. This is created because Git doesn’t allow storing empty folders. You can safely delete this file from your repo if you want.

Step 3 – Create your Multi-stage Build Pipeline

Now that we’ve got a repository we can create our mulit-stage build pipeline. This build pipeline will package the infrastructure files and store them and then perform a deployment. The multi-stage build pipeline is defined in a file called azure-pipelines.yml that we’ll put into the root folde of our repository.

  1. Click here to download a copy of the azure-pipelines.yml. You can see the content of this file here.
  2. Click the ellipsis (…) button next to the repository name and select Upload file(s).
  3. Click Browse and select the azure-pipelines.yml file you dowloaded.
  4. Set the Comment to something like “Added Pipeline Defnition”.
  5. Click Commit.ss_sonarqube_uploadpipelinefile
  6. Click Set up build button.
  7. Azure Pipelines will automatically detect the azure-pipelines.yml file in the root of our repository and configure our pipeline.
  8. Click the Run button. The build will fail because we haven’t yet created the service connection called Sonarqube-Azure to allow our pipeline to deploy to Azure. We also still still need to configure the parameters for the pipeline.ss_sonarqube_createbuildpipeline

Note: I’ll break down the contents of the azure-pipelines.yml at the end of this post so you get a feel for how a multi-stage build pipeline can be defined.

Step 4 – Create Service Connection to Azure

For Azure Pipelines to be able to deploy to Azure (or access other external services) it needs a service connection defined. In this step we’ll configure the service sonnection called Sonarqube-Azure that is referred to in the azure-pipelines.yml file. I won’t go into too much detail about what happens when we create a service connection as Azure Pipelines takes care of the details for you, but if you want to know more, read this page.

Important: This step assumes you have permissions to create service connections in the project and permissions to Azure to create a new Serivce Principal account with contributor permissions within the subscription. Many users won’t have this, so you might need to get a user with the enough permissions to the Azure subscription to do this step for you.

  1. Click the Project settings button in your project.
  2. Click Service connections under the Pipelines section.
  3. Click New service connection.
  4. Select Azure Resource Manager.
  5. Make sure Service Principal Authentication is selected.
  6. Enter Sonarqube-Azure for the Connection name. This must be exact, otherwise it won’t match the value in the azure-pipelines.yml file.
  7. Set Scope level to Subscription.
  8. From the Subscription box, select your Azure Subscription.
  9. Make sure the Resource group box is empty.
  10. Click OK.
  11. An authorization box will pop up requesting that you authenticate with the Azure subscription you want to deploy to.
  12. Enter the account details of a user who has permissions to create a Service Principal with contributor access to the subscription selected above.ss_sonarqube_createserviceconnection

You now have a service connection to Azure that any build pipeline (including the one we created earlier) in this project can use to deploy services to Azure.

Note: You can restrict the use of this Service connection by changing the Roles on the Service connection. See this page for more information.

Step 5 – Configure Pipeline Parameters

The ARM template contains a number of parameters which allow us to configure some of the things about the Azure resources we’re going to deploy, such as the Location (data center) to deploy to, the size of the resources and the site name our Sonarqube service will be exposed on.

In the azure-pipelines.yml file we configure the parameters that are passed to the ARM template from pipeline variables. Note: There are additional ARM template parameters that are exposed (such as sqlDatabaseSkuSizeGB and sonarqubeImageVersion), but we’ll leave the configuration of those parameters as a seprate exercise.

The parameters that are exposed as pipeline variables are:

  • siteName – The name of the web site. This will result in the Sonarqube web site being hosted at [siteName].azurewebsites.net.
  • sqlServerAdministratorUsername – The administrator username that will be used to administer this SQL database and for the Sonarqube front end to connct using. Note: for a Production service we should actually create another account for Sonarqube to use.
  • sqlServerAdministratorPassword – The password that will be used by Sonarqube to connect to the database.
  • servicePlanCapacity – The number of App Service plan nodes to use to run this Sonarqube service. Recommend leaving it at 1 unless you’ve got really heavy load.
  • servicePlanPricingTier – This is the App Service plan pricing tier to use for the service. Suggest S1 for testing, but for systems requiring greater performance then S2, S3, P1V2, P2V2 or P3V2.
  • sqlDatabaseSkuName – this is the performance of the SQL Server. There are a number of different performance options here and what you chose will need to depend on your load.
  • location – this is the code for the data center to deploy to. I use WestUS2, but chose whatever datacenter you wish.

The great thing is, you can change these variables at any time and then run your pipeline again and your infrastructure will be changed (scaled up/down/in/out) accordingly – without losing data. Note: You can’t change location or siteName after first deployment however.

To create your variables:

  1. Click Pipelines.
  2. Click the SonarqubeInAzure pipeline.
  3. Click the Edit button.
  4. Click the menu button (vertical ellipsis) and select Variables.
  5. Click the Add button and add the following parameters and values:
    • siteName – The globally unique name for your site. This will deploy the service to [siteName].azurewebsites.net. If this does not result in a globally unique name an error will occur during deployment.
    • sqlServerAdministratorUsername – Set to sonarqube.
    • sqlServerAdministratorPassword – Set to a strong password consisting of at least 8 characters including upper and lower case, numbers and symbols. Make sure you click the lock symbol to let Azure DevOps know this is a password and to treat it accordignly.
    • servicePlanCapacity – Set to 1 for now (you can always change and scale up later).
    • servicePlanPricingTier – Set to S1 for now (you can always change and scale up later).
    • sqlDatabaseSkuName – Set to GP_Gen5_2 for now (you can always change and scale up later). If you want to use the SQL Serverless database, use GP_S_Gen5_1, GP_S_Gen5_2 or GP_S_Gen5_4.
    • location – set to WestUS2 or whatever the code is for your preferred data center.
  6. You can also click the Settable at Queue time box against any of the parameters you want to be able to set when the job is manually queued.ss_sonarqube_createvariablesss_sonarqube_variables
  7. Click the Save and Queue button and select Save.

We are now ready to deploy our service by triggering the pipeline.

Step 6 – Run the Pipeline

The most common way an Azure Pipeline is going to get triggered is by committing a change to the repository the build pipeline is linked to. But in this case we are just going to trigger a manual build:

  1. Click Pipelines.
  2. Click the SonarqubeInAzure pipeline.
  3. Click the Run pipeline.
  4. Set any of the variables we want to change (for example if we wanted to scale up our services).
  5. Click Run.
  6. You can then watch the build and deploy stages complete.ss_sonarqube_runpipeline.gif

Your pipeline should have completed and your resources will be on thier way to being deployed to Azure. You can rerun this pipeline at any time with different variables to scale your services. You could even delete the front end app service completely and use this pipeline to redeploy the service again – saving lots of precious $$$.

Step 7 – Checkout your new Sonarqube Service

You can login to the Azure Portal to see the new resource group and resources that have been deployed.

  1. Open the Azure portal and log in.
  2. You will see a new resource group named [siteName]-rg.
  3. Open the [siteName]-rg.ss_sonarqube_resources
  4. Select the Web App with the name [siteName].ss_sonarqube_webapp
  5. Click the URL.
  6. Your Sonarqube application will open after a few seconds. Note: It may take a little while to load the first time depending on the performance you configured on your SQL database.ss_sonarqube_theapplication
  7. Login to Sonarqube using the username admin and the password admin. You’ll want to change this immediately.

You are now ready to use Sonarqube in your build pipelines.

Step 8 – Scaling your Sonarqube Services

One of the purposes of this process was to enable the resources to be scaled easily and non-desrtructively. All we need to do is:

  1. Click Pipelines.
  2. Click the SonarqubeInAzure pipeline.
  3. Click the Run pipeline.
  4. Set any of the variables we want to change to scale the service up/down/in/out.
  5. Click Run.
  6. You can then watch the build and deploy stages complete.

Of course you could do a lot of the scaling with Azure Automation, which is a better idea in the long term than using your build pipeline to scale the services because you’ll end up with hundreds of deployment records over time.

A Closer look at the Multi-stage Build Pipeline YAML

At the time of writing this post, the Multi-stage Build Pipeline YAML was relatively new and still in a preview state. This means that it is not fully documented. So, I’ll break down the file and highlight the interesting pieces:

Trigger

ss_sonarqube_yamltrigger

This section ensures the pipeline will only be triggered on changes to the master branch.

Stages

ss_sonarqube_yamlstages

This section contains the two stages: Build and Deploy. We could have as many stages as we like. For example: Build, Deploy Test, Deploy Prod.

Build Stage

ss_sonarqube_yamlbuildstage

This defines the steps to run in the build stage. It also requires the execution of the stage on an Azure DevOps agent in the vs2017-win2016 pool.

Build Stage Checkout Step

ss_sonarqube_yamlbuildcheckout

This step causes the repository to be checked out onto the Azure DevOps agent.

Build Stage Publish Artifacts Step

ss_sonarqube_yamlbuildpublish

This step takes the infrastructure folder from the checked out repository and stores it as an artifact that will always be accessible as long as the build record is stored. The artifact will also be made available to the next stage (the Deploy Stage). The purpose of this step is to ensure we have an immutable artifact available that we could always use to redeploy this exact build.

Deploy Stage

ss_sonarqube_yamldeploystage

The deploy stage takes the artifact produced in the build stage and deploys it. It runs on an Azure DevOps agent in the vs2017-win2016 pool.

It also specifies that this is a deployment to an environment called “dev“. This will cause the environment to show up in the environments section under pipelines in Azure DevOps.

ss_sonarqube_environments.png

The strategy and runOnce define that this deployment should only execute once each time the pipeline is triggered.

Deploy Stage Azure Resource Group Deployment Step

ss_sonarqube_yamldeploystep

This deploy step takes the ARM template from the infrastructure artifact and deploys it to the Sonarqube-Azure Service connection. It overrides the parameters (using the overrideParameters) property using build variables (e.g. $(siteName), $(servicePlanCapacity)).

But what about Azure Blueprints?

One final thing to consider: this deployment could be a great use case for implementing with Azure Blueprints. I would strongly suggest taking a look at using your build pipeline to deploy an Azure Blueprint containing the ARM template above.

Thank you very much for reading this and I hope you found it interesting.