Output connection strings and keys from Azure Bicep

If we're provisioning resources in Azure with Bicep, we may have a need to acquire the connection strings and keys of our newly deployed infrastructure. For example, the connection strings of an event hub or the access keys of a storage account. Perhaps we'd like to use them to run an end-to-end test, perhaps we'd like to store these secrets somewhere for later consumption. This post shows how to do that using Bicep and the listKeys helper. Optionally it shows how we could consume this in Azure Pipelines.

Please note that exporting keys / connection strings etc from Bicep / ARM templates is generally considered to be a less secure approach. This is because these values will be visible inside the deployments section of the Azure Portal. Anyone who has access to this will be able to see them. An alternative approach would be permissioning our pipeline to access the resources directly. You can read about that approach here.

Event Hub connection string

First of all, let's provision an Azure Event Hub. This involves deploying an event hub namespace, an event hub in that namespace and an authorization rule. The following Bicep will do this for us:

// Create an event hub namespace

var eventHubNamespaceName = 'evhns-demo'

resource eventHubNamespace 'Microsoft.EventHub/namespaces@2021-01-01-preview' = {
  name: eventHubNamespaceName
  location: resourceGroup().location
  sku: {
    name: 'Standard'
    tier: 'Standard'
    capacity: 1
  }
  properties: {
    zoneRedundant: true
  }
}

// Create an event hub inside the namespace

var eventHubName = 'evh-demo'

resource eventHubNamespaceName_eventHubName 'Microsoft.EventHub/namespaces/eventhubs@2021-01-01-preview' = {
  parent: eventHubNamespace
  name: eventHubName
  properties: {
    messageRetentionInDays: 7
    partitionCount: 1
  }
}

// Grant Listen and Send on our event hub

resource eventHubNamespaceName_eventHubName_ListenSend 'Microsoft.EventHub/namespaces/eventhubs/authorizationRules@2021-01-01-preview' = {
  parent: eventHubNamespaceName_eventHubName
  name: 'ListenSend'
  properties: {
    rights: [
      'Listen'
      'Send'
    ]
  }
  dependsOn: [
    eventHubNamespace
  ]
}

When this is deployed to Azure, it will result in creating something like this:

As we can see, there are connection strings available which can be used to access the event hub. How do we get a connection string that we can play with? It's easily achieved by appending the following to our Bicep:

// Determine our connection string

var eventHubNamespaceConnectionString = listKeys(eventHubNamespaceName_eventHubName_ListenSend.id, eventHubNamespaceName_eventHubName_ListenSend.apiVersion).primaryConnectionString

// Output our variables

output eventHubNamespaceConnectionString string = eventHubNamespaceConnectionString
output eventHubName string = eventHubName

What we're doing here is using the listKeys helper on our authorization rule and retrieving the handy primaryConnectionString, which is then exposed as an output variable.

Storage Account connection string

We'd like to obtain a connection string for a storage account also. Let's put together a Bicep file that creates a storage account and a container therein. (Incidentally, it's fairly common to have a storage account provisioned alongside an event hub to facilitate reading from an event hub.)

// Create a storage account

var storageAccountName = 'stdemo'

resource eventHubNamespaceName_storageAccount 'Microsoft.Storage/storageAccounts@2021-02-01' = {
  name: storageAccountName
  location: resourceGroup().location
  sku: {
    name: 'Standard_LRS'
    tier: 'Standard'
  }
  kind: 'StorageV2'
  properties: {
    networkAcls: {
      bypass: 'AzureServices'
      defaultAction: 'Allow'
    }
    accessTier: 'Hot'
    allowBlobPublicAccess: false
    minimumTlsVersion: 'TLS1_2'
    allowSharedKeyAccess: true
  }
}

// create a container inside that storage account

var blobContainerName = 'test-container'

resource storageAccountName_default_containerName 'Microsoft.Storage/storageAccounts/blobServices/containers@2021-02-01' = {
  name: '${storageAccountName}/default/${blobContainerName}'
  dependsOn: [
    eventHubNamespaceName_storageAccount
  ]
}

When this is deployed to Azure, it will result in creating something like this:

Again we can see, there are connection strings available in the Azure Portal, which can be used to access the storage account. However, things aren't quite as simple as previously; in that there doesn't seem to be a way to directly acquire a connection string. What we can do, is acquire a key; and construct ourselves a connection string with that. Here's how:

// Determine our connection string

var blobStorageConnectionString = 'DefaultEndpointsProtocol=https;AccountName=${eventHubNamespaceName_storageAccount.name};EndpointSuffix=${environment().suffixes.storage};AccountKey=${listKeys(eventHubNamespaceName_storageAccount.id, eventHubNamespaceName_storageAccount.apiVersion).keys[0].value}'

// Output our variable

output blobStorageConnectionString string = blobStorageConnectionString
output blobContainerName string = blobContainerName

If you just wanted to know how to acquire connection strings from Bicep then you can stop now; we're done! But if you're curious on how the Bicep might connect to ~~the shoulder~~ Azure Pipelines… Read on.

From Bicep to Azure Pipelines

If we put together our snippets above into a single Bicep file it would look like this:

// Create an event hub namespace

var eventHubNamespaceName = 'evhns-demo'

resource eventHubNamespace 'Microsoft.EventHub/namespaces@2021-01-01-preview' = {
  name: eventHubNamespaceName
  location: resourceGroup().location
  sku: {
    name: 'Standard'
    tier: 'Standard'
    capacity: 1
  }
  properties: {
    zoneRedundant: true
  }
}

// Create an event hub inside the namespace

var eventHubName = 'evh-demo'

resource eventHubNamespaceName_eventHubName 'Microsoft.EventHub/namespaces/eventhubs@2021-01-01-preview' = {
  parent: eventHubNamespace
  name: eventHubName
  properties: {
    messageRetentionInDays: 7
    partitionCount: 1
  }
}

// Grant Listen and Send on our event hub

resource eventHubNamespaceName_eventHubName_ListenSend 'Microsoft.EventHub/namespaces/eventhubs/authorizationRules@2021-01-01-preview' = {
  parent: eventHubNamespaceName_eventHubName
  name: 'ListenSend'
  properties: {
    rights: [
      'Listen'
      'Send'
    ]
  }
  dependsOn: [
    eventHubNamespace
  ]
}

// Create a storage account

var storageAccountName = 'stdemo'

resource eventHubNamespaceName_storageAccount 'Microsoft.Storage/storageAccounts@2021-02-01' = {
  name: storageAccountName
  location: resourceGroup().location
  sku: {
    name: 'Standard_LRS'
    tier: 'Standard'
  }
  kind: 'StorageV2'
  properties: {
    networkAcls: {
      bypass: 'AzureServices'
      defaultAction: 'Allow'
    }
    accessTier: 'Hot'
    allowBlobPublicAccess: false
    minimumTlsVersion: 'TLS1_2'
    allowSharedKeyAccess: true
  }
}

// create a container inside that storage account

var blobContainerName = 'test-container'

resource storageAccountName_default_containerName 'Microsoft.Storage/storageAccounts/blobServices/containers@2021-02-01' = {
  name: '${storageAccountName}/default/${blobContainerName}'
  dependsOn: [
    eventHubNamespaceName_storageAccount
  ]
}

// Determine our connection strings

var blobStorageConnectionString       = 'DefaultEndpointsProtocol=https;AccountName=${eventHubNamespaceName_storageAccount.name};EndpointSuffix=${environment().suffixes.storage};AccountKey=${listKeys(eventHubNamespaceName_storageAccount.id, eventHubNamespaceName_storageAccount.apiVersion).keys[0].value}'
var eventHubNamespaceConnectionString = listKeys(eventHubNamespaceName_eventHubName_ListenSend.id, eventHubNamespaceName_eventHubName_ListenSend.apiVersion).primaryConnectionString

// Output our variables

output blobStorageConnectionString string = blobStorageConnectionString
output blobContainerName string = blobContainerName
output eventHubNamespaceConnectionString string = eventHubNamespaceConnectionString
output eventHubName string = eventHubName

This might be consumed in an Azure Pipeline that looks like this:

- bash: az bicep build --file infra/our-test-app/main.bicep
  displayName: 'Compile Bicep to ARM'

- task: AzureResourceManagerTemplateDeployment@3
  name: DeploySharedInfra
  displayName: Deploy Shared ARM Template
  inputs:
    deploymentScope: Resource Group
    azureResourceManagerConnection: ${{ parameters.serviceConnection }}
    subscriptionId: $(subscriptionId)
    action: Create Or Update Resource Group
    resourceGroupName: $(azureResourceGroup)
    location: $(location)
    templateLocation: Linked artifact
    csmFile: 'infra/our-test-app/main.json' # created by bash script
    deploymentMode: Incremental
    deploymentOutputs: deployOutputs

- task: PowerShell@2
  name: 'SetOutputVariables'
  displayName: 'Set Output Variables'
  inputs:
    targetType: inline
    script: |
      $armOutputObj = '$(deployOutputs)' | ConvertFrom-Json
      $armOutputObj.PSObject.Properties | ForEach-Object {
        $keyname = $_.Name
        $value = $_.Value.value

        # Creates a standard pipeline variable
        Write-Output "##vso[task.setvariable variable=$keyName;]$value"

        # Creates an output variable
        Write-Output "##vso[task.setvariable variable=$keyName;issecret=true;isOutput=true]$value"

        # Display keys in pipeline
        Write-Output "output variable: $keyName"
      }
    pwsh: true

Above we can see:

• the Bicep get compiled to ARM
• the ARM is deployed to Azure, with deploymentOutputs being passed out at the end
• the outputs are turned into secret output variables inside the pipeline (the names of which are printed to the console)

With the above in place, we now have all of our variables in place; blobStorageConnectionString, blobContainerName, eventHubNamespaceConnectionString and eventHubName. These could now be consumed in whatever way is useful. Consider the following:

- task: UseDotNet@2
  displayName: 'Install .NET Core SDK 3.1.x'
  inputs:
    packageType: 'sdk'
    version: 3.1.x

- task: DotNetCoreCLI@2
  displayName: 'dotnet run eventhub test'
  inputs:
    command: 'run'
    arguments: 'eventhub test --eventHubNamespaceConnectionString "$(eventHubNamespaceConnectionString)" --eventHubName "$(eventHubName)" --blobStorageConnectionString "$(blobStorageConnectionString)" --blobContainerName "$(blobContainerName)"'
    workingDirectory: '$(Build.SourcesDirectory)/OurTestApp'

Here we run a .NET application and pass it our connection strings. Please note, there's nothing .NET specific about what we're doing above - it could be any kind of application, bash script or similar that consumes our connection strings. The significant thing is that we can acquire connection strings in an automated fashion, for use in whichever manner pleases us.

Output connection strings and keys from Azure Bicep

If you're provisioning resources in Azure with Bicep, you may have a need to acquire the connection strings and keys of your newly deployed infrastructure. For example, the connection strings of an event hub or the access keys of a storage account. Perhaps you'd like to use them to run an end-to-end test, perhaps you'd like to store these secrets somewhere for later consumption. This post shows how to do that using Bicep and the listKeys helper. Optionally it shows how you could consume this in Azure Pipelines.

Event Hub connection string

First of all, let's provision an Azure Event Hub. This involves deploying an event hub namespace, an event hub in that namespace and an authorization rule. The following Bicep will do this for us:

// Create an event hub namespace

var eventHubNamespaceName = 'evhns-demo'

resource eventHubNamespace 'Microsoft.EventHub/namespaces@2021-01-01-preview' = {
  name: eventHubNamespaceName
  location: resourceGroup().location
  sku: {
    name: 'Standard'
    tier: 'Standard'
    capacity: 1
  }
  properties: {
    zoneRedundant: true
  }
}

// Create an event hub inside the namespace

var eventHubName = 'evh-demo'

resource eventHubNamespaceName_eventHubName 'Microsoft.EventHub/namespaces/eventhubs@2021-01-01-preview' = {
  parent: eventHubNamespace
  name: eventHubName
  properties: {
    messageRetentionInDays: 7
    partitionCount: 1
  }
}

// Grant Listen and Send on our event hub

resource eventHubNamespaceName_eventHubName_ListenSend 'Microsoft.EventHub/namespaces/eventhubs/authorizationRules@2021-01-01-preview' = {
  parent: eventHubNamespaceName_eventHubName
  name: 'ListenSend'
  properties: {
    rights: [
      'Listen'
      'Send'
    ]
  }
  dependsOn: [
    eventHubNamespace
  ]
}

When this is deployed to Azure, it will result in creating something like this:

As we can see, there are connection strings available which can be used to access the event hub. How do we get a connection string that we can play with? It's easily achieved by appending the following to our Bicep:

// Determine our connection string

var eventHubNamespaceConnectionString = listKeys(eventHubNamespaceName_eventHubName_ListenSend.id, eventHubNamespaceName_eventHubName_ListenSend.apiVersion).primaryConnectionString 

// Output our variables

output eventHubNamespaceConnectionString string = eventHubNamespaceConnectionString
output eventHubName string = eventHubName

What we're doing here is using the listKeys helper on our authorization rule and retrieve the handy primaryConnectionString, which is then exposed as an output variable.

Storage Account connection string

We'd like to obtain a connection string for a storage account also. Let's put together a Bicep file that creates a storage account and a container therein. (Incidentally, it's fairly common to have a storage account provisioned alongside an event hub to facilitate reading from an event hub.)

// Create a storage account

var storageAccountName = 'stdemo'

resource eventHubNamespaceName_storageAccount 'Microsoft.Storage/storageAccounts@2021-02-01' = {
  name: storageAccountName
  location: resourceGroup().location
  sku: {
    name: 'Standard_LRS'
    tier: 'Standard'
  }
  kind: 'StorageV2'
  properties: {
    networkAcls: {
      bypass: 'AzureServices'
      defaultAction: 'Allow'
    }
    accessTier: 'Hot'
    allowBlobPublicAccess: false
    minimumTlsVersion: 'TLS1_2'
    allowSharedKeyAccess: true
  }
}

// create a container inside that storage account

var blobContainerName = 'test-container'

resource storageAccountName_default_containerName 'Microsoft.Storage/storageAccounts/blobServices/containers@2021-02-01' = {
  name: '${storageAccountName}/default/${blobContainerName}'
  dependsOn: [
    eventHubNamespaceName_storageAccount
  ]
}

When this is deployed to Azure, it will result in creating something like this:

Again we can see, there are connection strings available in the Azure Portal, which can be used to access the storage account. However, things aren't quite as simple as previously; in that there doesn't seem to be a way to directly acquire a connection string. What we can do, is acquire a key; and construct ourselves a connection string with that. Here's how:

// Determine our connection string

var blobStorageConnectionString = 'DefaultEndpointsProtocol=https;AccountName=${eventHubNamespaceName_storageAccount.name};EndpointSuffix=${environment().suffixes.storage};AccountKey=${listKeys(eventHubNamespaceName_storageAccount.id, eventHubNamespaceName_storageAccount.apiVersion).keys[0].value}'

// Output our variable

output blobStorageConnectionString string = blobStorageConnectionString
output blobContainerName string = blobContainerName

If you just wanted to know how to acquire connection strings from Bicep then you can stop now; we're done! But if you're curious on how the Bicep might connect to ~~the shoulder~~ Azure Pipelines… Read on.

From Bicep to Azure Pipelines

If we put together our snippets above into a single Bicep file it would look like this:

// Create an event hub namespace

var eventHubNamespaceName = 'evhns-demo'

resource eventHubNamespace 'Microsoft.EventHub/namespaces@2021-01-01-preview' = {
  name: eventHubNamespaceName
  location: resourceGroup().location
  sku: {
    name: 'Standard'
    tier: 'Standard'
    capacity: 1
  }
  properties: {
    zoneRedundant: true
  }
}

// Create an event hub inside the namespace

var eventHubName = 'evh-demo'

resource eventHubNamespaceName_eventHubName 'Microsoft.EventHub/namespaces/eventhubs@2021-01-01-preview' = {
  parent: eventHubNamespace
  name: eventHubName
  properties: {
    messageRetentionInDays: 7
    partitionCount: 1
  }
}

// Grant Listen and Send on our event hub

resource eventHubNamespaceName_eventHubName_ListenSend 'Microsoft.EventHub/namespaces/eventhubs/authorizationRules@2021-01-01-preview' = {
  parent: eventHubNamespaceName_eventHubName
  name: 'ListenSend'
  properties: {
    rights: [
      'Listen'
      'Send'
    ]
  }
  dependsOn: [
    eventHubNamespace
  ]
}

// Create a storage account

var storageAccountName = 'stdemo'

resource eventHubNamespaceName_storageAccount 'Microsoft.Storage/storageAccounts@2021-02-01' = {
  name: storageAccountName
  location: resourceGroup().location
  sku: {
    name: 'Standard_LRS'
    tier: 'Standard'
  }
  kind: 'StorageV2'
  properties: {
    networkAcls: {
      bypass: 'AzureServices'
      defaultAction: 'Allow'
    }
    accessTier: 'Hot'
    allowBlobPublicAccess: false
    minimumTlsVersion: 'TLS1_2'
    allowSharedKeyAccess: true
  }
}

// create a container inside that storage account

var blobContainerName = 'test-container'

resource storageAccountName_default_containerName 'Microsoft.Storage/storageAccounts/blobServices/containers@2021-02-01' = {
  name: '${storageAccountName}/default/${blobContainerName}'
  dependsOn: [
    eventHubNamespaceName_storageAccount
  ]
}

// Determine our connection strings

var blobStorageConnectionString       = 'DefaultEndpointsProtocol=https;AccountName=${eventHubNamespaceName_storageAccount.name};EndpointSuffix=${environment().suffixes.storage};AccountKey=${listKeys(eventHubNamespaceName_storageAccount.id, eventHubNamespaceName_storageAccount.apiVersion).keys[0].value}'
var eventHubNamespaceConnectionString = listKeys(eventHubNamespaceName_eventHubName_ListenSend.id, eventHubNamespaceName_eventHubName_ListenSend.apiVersion).primaryConnectionString 

// Output our variables

output blobStorageConnectionString string = blobStorageConnectionString
output blobContainerName string = blobContainerName
output eventHubNamespaceConnectionString string = eventHubNamespaceConnectionString
output eventHubName string = eventHubName

This might be consumed in an Azure Pipeline that looks like this:

    - bash: az bicep build --file infra/our-test-app/main.bicep
      displayName: "Compile Bicep to ARM"

    - task: AzureResourceManagerTemplateDeployment@3
      name: DeploySharedInfra
      displayName: Deploy Shared ARM Template
      inputs:
        deploymentScope: Resource Group
        azureResourceManagerConnection: ${{ parameters.serviceConnection }}
        subscriptionId: $(subscriptionId)
        action: Create Or Update Resource Group
        resourceGroupName: $(azureResourceGroup)
        location: $(location)
        templateLocation: Linked artifact
        csmFile: 'infra/our-test-app/main.json' # created by bash script
        deploymentMode: Incremental
        deploymentOutputs: deployOutputs

    - task: PowerShell@2
      name: 'SetOutputVariables'
      displayName: "Set Output Variables"
      inputs:
        targetType: inline
        script: |
          $armOutputObj = '$(deployOutputs)' | ConvertFrom-Json
          $armOutputObj.PSObject.Properties | ForEach-Object {
            $keyname = $_.Name
            $value = $_.Value.value

            # Creates a standard pipeline variable
            Write-Output "##vso[task.setvariable variable=$keyName;]$value"

            # Creates an output variable
            Write-Output "##vso[task.setvariable variable=$keyName;issecret=true;isOutput=true]$value"

            # Display keys in pipeline
            Write-Output "output variable: $keyName"
          }
        pwsh: true

Above we can see:

• the Bicep get compiled to ARM
• the ARM is deployed to Azure, with deploymentOutputs being passed out at the end
• the outputs are turned into secret output variables inside the pipeline (the names of which are printed to the console)

With the above in place, we now have all of our variables in place; blobStorageConnectionString, blobContainerName, eventHubNamespaceConnectionString and eventHubName. These could now be consumed in whatever way is useful. Consider the following:

    - task: UseDotNet@2
      displayName: 'Install .NET Core SDK 3.1.x'
      inputs:
        packageType: 'sdk'
        version: 3.1.x

    - task: DotNetCoreCLI@2
      displayName: "dotnet run eventhub test"
      inputs:
        command: 'run'
        arguments: 'eventhub test --eventHubNamespaceConnectionString "$(eventHubNamespaceConnectionString)" --eventHubName "$(eventHubName)" --blobStorageConnectionString "$(blobStorageConnectionString)" --blobContainerName "$(blobContainerName)"'
        workingDirectory: '$(Build.SourcesDirectory)/OurTestApp'

Here we run a .NET application and pass it our connection strings. Please note, there's nothing .NET specific about what we're doing above - it could be any kind of application, bash script or similar that consumes our connection strings. The significant thing is that we can acquire connection strings in an automated fashion, for use in whichever manner pleases us.

Output connection strings and keys from Azure Bicep

If you're provisioning resources in Azure with Bicep, you may have a need to acquire the connection strings and keys of your newly deployed infrastructure. For example, the connection strings of an event hub or the access keys of a storage account. Perhaps you'd like to use them to run an end-to-end test, perhaps you'd like to store these secrets somewhere for later consumption. This post shows how to do that using Bicep and the listKeys helper. Optionally it shows how you could consume this in Azure Pipelines.

Event Hub connection string

First of all, let's provision an Azure Event Hub. This involves deploying an event hub namespace, an event hub in that namespace and an authorization rule. The following Bicep will do this for us:

// Create an event hub namespace

var eventHubNamespaceName = 'evhns-demo'

resource eventHubNamespace 'Microsoft.EventHub/namespaces@2021-01-01-preview' = {
  name: eventHubNamespaceName
  location: resourceGroup().location
  sku: {
    name: 'Standard'
    tier: 'Standard'
    capacity: 1
  }
  properties: {
    zoneRedundant: true
  }
}

// Create an event hub inside the namespace

var eventHubName = 'evh-demo'

resource eventHubNamespaceName_eventHubName 'Microsoft.EventHub/namespaces/eventhubs@2021-01-01-preview' = {
  parent: eventHubNamespace
  name: eventHubName
  properties: {
    messageRetentionInDays: 7
    partitionCount: 1
  }
}

// Grant Listen and Send on our event hub

resource eventHubNamespaceName_eventHubName_ListenSend 'Microsoft.EventHub/namespaces/eventhubs/authorizationRules@2021-01-01-preview' = {
  parent: eventHubNamespaceName_eventHubName
  name: 'ListenSend'
  properties: {
    rights: [
      'Listen'
      'Send'
    ]
  }
  dependsOn: [
    eventHubNamespace
  ]
}

When this is deployed to Azure, it will result in creating something like this:

As we can see, there are connection strings available which can be used to access the event hub. How do we get a connection string that we can play with? It's easily achieved by appending the following to our Bicep:

// Determine our connection string

var eventHubNamespaceConnectionString = listKeys(eventHubNamespaceName_eventHubName_ListenSend.id, eventHubNamespaceName_eventHubName_ListenSend.apiVersion).primaryConnectionString 

// Output our variables

output eventHubNamespaceConnectionString string = eventHubNamespaceConnectionString
output eventHubName string = eventHubName

What we're doing here is using the listKeys helper on our authorization rule and retrieve the handy primaryConnectionString, which is then exposed as an output variable.

Storage Account connection string

We'd like to obtain a connection string for a storage account also. Let's put together a Bicep file that creates a storage account and a container therein. (Incidentally, it's fairly common to have a storage account provisioned alongside an event hub to facilitate reading from an event hub.)

// Create a storage account

var storageAccountName = 'stdemo'

resource eventHubNamespaceName_storageAccount 'Microsoft.Storage/storageAccounts@2021-02-01' = {
  name: storageAccountName
  location: resourceGroup().location
  sku: {
    name: 'Standard_LRS'
    tier: 'Standard'
  }
  kind: 'StorageV2'
  properties: {
    networkAcls: {
      bypass: 'AzureServices'
      defaultAction: 'Allow'
    }
    accessTier: 'Hot'
    allowBlobPublicAccess: false
    minimumTlsVersion: 'TLS1_2'
    allowSharedKeyAccess: true
  }
}

// create a container inside that storage account

var blobContainerName = 'test-container'

resource storageAccountName_default_containerName 'Microsoft.Storage/storageAccounts/blobServices/containers@2021-02-01' = {
  name: '${storageAccountName}/default/${blobContainerName}'
  dependsOn: [
    eventHubNamespaceName_storageAccount
  ]
}

When this is deployed to Azure, it will result in creating something like this:

Again we can see, there are connection strings available which can be used to access the storage account. However, things aren't quite as simple as previously; in that there doesn't seem to be directly acquire a connection string. What we can do, is acquire a key; and construct ourselves a connection string on that basis. Here's how:

// Determine our connection string

var blobStorageConnectionString = 'DefaultEndpointsProtocol=https;AccountName=${eventHubNamespaceName_storageAccount.name};EndpointSuffix=${environment().suffixes.storage};AccountKey=${listKeys(eventHubNamespaceName_storageAccount.id, eventHubNamespaceName_storageAccount.apiVersion).keys[0].value}'

// Output our variable

output blobStorageConnectionString string = blobStorageConnectionString
output blobContainerName string = blobContainerName

If you just wanted to know how to acquire connection strings from Bicep then you can stop now; we're done! But if you're curious on how the Bicep might connect to ~~the shoulder~~ Azure Pipelines… Read on.

From Bicep to Azure Pipelines

If we put together our snippets above into a single Bicep file it would look like this:

// Create an event hub namespace

var eventHubNamespaceName = 'evhns-demo'

resource eventHubNamespace 'Microsoft.EventHub/namespaces@2021-01-01-preview' = {
  name: eventHubNamespaceName
  location: resourceGroup().location
  sku: {
    name: 'Standard'
    tier: 'Standard'
    capacity: 1
  }
  properties: {
    zoneRedundant: true
  }
}

// Create an event hub inside the namespace

var eventHubName = 'evh-demo'

resource eventHubNamespaceName_eventHubName 'Microsoft.EventHub/namespaces/eventhubs@2021-01-01-preview' = {
  parent: eventHubNamespace
  name: eventHubName
  properties: {
    messageRetentionInDays: 7
    partitionCount: 1
  }
}

// Grant Listen and Send on our event hub

resource eventHubNamespaceName_eventHubName_ListenSend 'Microsoft.EventHub/namespaces/eventhubs/authorizationRules@2021-01-01-preview' = {
  parent: eventHubNamespaceName_eventHubName
  name: 'ListenSend'
  properties: {
    rights: [
      'Listen'
      'Send'
    ]
  }
  dependsOn: [
    eventHubNamespace
  ]
}

// Create a storage account

var storageAccountName = 'stdemo'

resource eventHubNamespaceName_storageAccount 'Microsoft.Storage/storageAccounts@2021-02-01' = {
  name: storageAccountName
  location: resourceGroup().location
  sku: {
    name: 'Standard_LRS'
    tier: 'Standard'
  }
  kind: 'StorageV2'
  properties: {
    networkAcls: {
      bypass: 'AzureServices'
      defaultAction: 'Allow'
    }
    accessTier: 'Hot'
    allowBlobPublicAccess: false
    minimumTlsVersion: 'TLS1_2'
    allowSharedKeyAccess: true
  }
}

// create a container inside that storage account

var blobContainerName = 'test-container'

resource storageAccountName_default_containerName 'Microsoft.Storage/storageAccounts/blobServices/containers@2021-02-01' = {
  name: '${storageAccountName}/default/${blobContainerName}'
  dependsOn: [
    eventHubNamespaceName_storageAccount
  ]
}

// Determine our connection strings

var blobStorageConnectionString       = 'DefaultEndpointsProtocol=https;AccountName=${eventHubNamespaceName_storageAccount.name};EndpointSuffix=${environment().suffixes.storage};AccountKey=${listKeys(eventHubNamespaceName_storageAccount.id, eventHubNamespaceName_storageAccount.apiVersion).keys[0].value}'
var eventHubNamespaceConnectionString = listKeys(eventHubNamespaceName_eventHubName_ListenSend.id, eventHubNamespaceName_eventHubName_ListenSend.apiVersion).primaryConnectionString 

// Output our variables

output blobStorageConnectionString string = blobStorageConnectionString
output blobContainerName string = blobContainerName
output eventHubNamespaceConnectionString string = eventHubNamespaceConnectionString
output eventHubName string = eventHubName

This might be consumed in an Azure Pipeline that looks like this:

    - bash: az bicep build --file infra/our-test-app/main.bicep
      displayName: "Compile Bicep to ARM"

    - task: AzureResourceManagerTemplateDeployment@3
      name: DeploySharedInfra
      displayName: Deploy Shared ARM Template
      inputs:
        deploymentScope: Resource Group
        azureResourceManagerConnection: ${{ parameters.serviceConnection }}
        subscriptionId: $(subscriptionId)
        action: Create Or Update Resource Group
        resourceGroupName: $(azureResourceGroup)
        location: $(location)
        templateLocation: Linked artifact
        csmFile: 'infra/our-test-app/main.json' # created by bash script
        deploymentMode: Incremental
        deploymentOutputs: deployOutputs

    - task: PowerShell@2
      name: 'SetOutputVariables'
      displayName: "Set Output Variables"
      inputs:
        targetType: inline
        script: |
          $armOutputObj = '$(deployOutputs)' | ConvertFrom-Json
          $armOutputObj.PSObject.Properties | ForEach-Object {
            $keyname = $_.Name
            $value = $_.Value.value

            # Creates a standard pipeline variable
            Write-Output "##vso[task.setvariable variable=$keyName;]$value"

            # Creates an output variable
            Write-Output "##vso[task.setvariable variable=$keyName;issecret=true;isOutput=true]$value"

            # Display keys in pipeline
            Write-Output "output variable: $keyName"
          }
        pwsh: true

Above we can see:

• the Bicep get compiled to ARM
• the ARM is deployed to Azure, with deploymentOutputs being passed out at the end
• the outputs are turned into secret output variables inside the pipeline (the names of which are printed to the console)

With the above in place, we now have all of our variables in place; blobStorageConnectionString, blobContainerName, eventHubNamespaceConnectionString and eventHubName. These could now be consumed in whatever way is useful. Consider the following:

    - task: UseDotNet@2
      displayName: 'Install .NET Core SDK 3.1.x'
      inputs:
        packageType: 'sdk'
        version: 3.1.x

    - task: DotNetCoreCLI@2
      displayName: "dotnet run eventhub test"
      inputs:
        command: 'run'
        arguments: 'eventhub test --eventHubNamespaceConnectionString "$(eventHubNamespaceConnectionString)" --eventHubName "$(eventHubName)" --blobStorageConnectionString "$(blobStorageConnectionString)" --blobContainerName "$(blobContainerName)"'
        workingDirectory: '$(Build.SourcesDirectory)/OurTestApp'

Here we run a .NET application and pass it our connection strings. Please note, there's nothing .NET specific about what we're doing above - it could be any kind of application, bash script or similar that consumes our connection strings. The significant thing is that we can acquire connection strings in an automated fashion, for use in whichever manner pleases us.

Output connection strings and keys from Azure Bicep

If you're provisioning resources in Azure with Bicep, you may have a need to acquire the connection strings and keys of your newly deployed infrastructure. For example, the connection strings of an event hub or the access keys of a storage account. Perhaps you'd like to use them to run an end-to-end test, perhaps you'd like to store these secrets somewhere for later consumption. This post shows how to do that using Bicep and the listKeys helper. Optionally it shows how you could consume this in Azure Pipelines.

Event Hub connection string

First of all, let's provision an Azure Event Hub. This involves deploying an event hub namespace, an event hub in that namespace and an authorization rule. The following Bicep will do this for us:

// Create an event hub namespace

var eventHubNamespaceName = 'evhns-demo'

resource eventHubNamespace 'Microsoft.EventHub/namespaces@2021-01-01-preview' = {
  name: eventHubNamespaceName
  location: resourceGroup().location
  sku: {
    name: 'Standard'
    tier: 'Standard'
    capacity: 1
  }
  properties: {
    zoneRedundant: true
  }
}

// Create an event hub inside the namespace

var eventHubName = 'evh-demo'

resource eventHubNamespaceName_eventHubName 'Microsoft.EventHub/namespaces/eventhubs@2021-01-01-preview' = {
  parent: eventHubNamespace
  name: eventHubName
  properties: {
    messageRetentionInDays: 7
    partitionCount: 1
  }
}

// Grant Listen and Send on our event hub

resource eventHubNamespaceName_eventHubName_ListenSend 'Microsoft.EventHub/namespaces/eventhubs/authorizationRules@2021-01-01-preview' = {
  parent: eventHubNamespaceName_eventHubName
  name: 'ListenSend'
  properties: {
    rights: [
      'Listen'
      'Send'
    ]
  }
  dependsOn: [
    eventHubNamespace
  ]
}

When this is deployed to Azure, it will result in creating something like this:

As we can see, there are connection strings available which can be used to access the event hub. How do we get a connection string that we can play with? It's easily achieved by appending the following to our Bicep:

// Determine our connection string

var eventHubNamespaceConnectionString = listKeys(eventHubNamespaceName_eventHubName_ListenSend.id, eventHubNamespaceName_eventHubName_ListenSend.apiVersion).primaryConnectionString 

// Output our variables

output eventHubNamespaceConnectionString string = eventHubNamespaceConnectionString
output eventHubName string = eventHubName

What we're doing here is using the listKeys helper on our authorization rule and retrieve the handy primaryConnectionString, which is then exposed as an output variable.

Storage Account connection string

We'd like to obtain a connection string for a storage account also. Let's put together a Bicep file that creates a storage account and a container therein. (Incidentally, it's fairly common to have a storage account provisioned alongside an event hub to facilitate reading from an event hub.)

// Create a storage account

var storageAccountName = 'stdemo'

resource eventHubNamespaceName_storageAccount 'Microsoft.Storage/storageAccounts@2021-02-01' = {
  name: storageAccountName
  location: resourceGroup().location
  sku: {
    name: 'Standard_LRS'
    tier: 'Standard'
  }
  kind: 'StorageV2'
  properties: {
    networkAcls: {
      bypass: 'AzureServices'
      defaultAction: 'Allow'
    }
    accessTier: 'Hot'
    allowBlobPublicAccess: false
    minimumTlsVersion: 'TLS1_2'
    allowSharedKeyAccess: true
  }
}

// create a container inside that storage account

var blobContainerName = 'test-container'

resource storageAccountName_default_containerName 'Microsoft.Storage/storageAccounts/blobServices/containers@2021-02-01' = {
  name: '${storageAccountName}/default/${blobContainerName}'
  dependsOn: [
    eventHubNamespaceName_storageAccount
  ]
}

When this is deployed to Azure, it will result in creating something like this:

Again we can see, there are connection strings available which can be used to access the storage account. However, things aren't quite as simple as previously; in that there doesn't seem to be directly acquire a connection string. What we can do, is acquire a key; and construct ourselves a connection string on that basis. Here's how:

// Determine our connection string

var blobStorageConnectionString = 'DefaultEndpointsProtocol=https;AccountName=${eventHubNamespaceName_storageAccount.name};EndpointSuffix=${environment().suffixes.storage};AccountKey=${listKeys(eventHubNamespaceName_storageAccount.id, eventHubNamespaceName_storageAccount.apiVersion).keys[0].value}'

// Output our variable

output blobStorageConnectionString string = blobStorageConnectionString
output blobContainerName string = blobContainerName

If you just wanted to know how to acquire connection strings from Bicep then you can stop now; we're done! But if you're curious on how the Bicep might connect to ~~the shoulder~~ Azure Pipelines… Read on.

From Bicep to Azure Pipelines

If we put together our snippets above into a single Bicep file it would look like this:

// Create an event hub namespace

var eventHubNamespaceName = 'evhns-demo'

resource eventHubNamespace 'Microsoft.EventHub/namespaces@2021-01-01-preview' = {
  name: eventHubNamespaceName
  location: resourceGroup().location
  sku: {
    name: 'Standard'
    tier: 'Standard'
    capacity: 1
  }
  properties: {
    zoneRedundant: true
  }
}

// Create an event hub inside the namespace

var eventHubName = 'evh-demo'

resource eventHubNamespaceName_eventHubName 'Microsoft.EventHub/namespaces/eventhubs@2021-01-01-preview' = {
  parent: eventHubNamespace
  name: eventHubName
  properties: {
    messageRetentionInDays: 7
    partitionCount: 1
  }
}

// Grant Listen and Send on our event hub

resource eventHubNamespaceName_eventHubName_ListenSend 'Microsoft.EventHub/namespaces/eventhubs/authorizationRules@2021-01-01-preview' = {
  parent: eventHubNamespaceName_eventHubName
  name: 'ListenSend'
  properties: {
    rights: [
      'Listen'
      'Send'
    ]
  }
  dependsOn: [
    eventHubNamespace
  ]
}

// Create a storage account

var storageAccountName = 'stdemo'

resource eventHubNamespaceName_storageAccount 'Microsoft.Storage/storageAccounts@2021-02-01' = {
  name: storageAccountName
  location: resourceGroup().location
  sku: {
    name: 'Standard_LRS'
    tier: 'Standard'
  }
  kind: 'StorageV2'
  properties: {
    networkAcls: {
      bypass: 'AzureServices'
      defaultAction: 'Allow'
    }
    accessTier: 'Hot'
    allowBlobPublicAccess: false
    minimumTlsVersion: 'TLS1_2'
    allowSharedKeyAccess: true
  }
}

// create a container inside that storage account

var blobContainerName = 'test-container'

resource storageAccountName_default_containerName 'Microsoft.Storage/storageAccounts/blobServices/containers@2021-02-01' = {
  name: '${storageAccountName}/default/${blobContainerName}'
  dependsOn: [
    eventHubNamespaceName_storageAccount
  ]
}

// Determine our connection strings

var blobStorageConnectionString       = 'DefaultEndpointsProtocol=https;AccountName=${eventHubNamespaceName_storageAccount.name};EndpointSuffix=${environment().suffixes.storage};AccountKey=${listKeys(eventHubNamespaceName_storageAccount.id, eventHubNamespaceName_storageAccount.apiVersion).keys[0].value}'
var eventHubNamespaceConnectionString = listKeys(eventHubNamespaceName_eventHubName_ListenSend.id, eventHubNamespaceName_eventHubName_ListenSend.apiVersion).primaryConnectionString 

// Output our variables

output blobStorageConnectionString string = blobStorageConnectionString
output blobContainerName string = blobContainerName
output eventHubNamespaceConnectionString string = eventHubNamespaceConnectionString
output eventHubName string = eventHubName

This might be consumed in an Azure Pipeline that looks like this:

    - bash: az bicep build --file infra/our-test-app/main.bicep
      displayName: "Compile Bicep to ARM"

    - task: AzureResourceManagerTemplateDeployment@3
      name: DeploySharedInfra
      displayName: Deploy Shared ARM Template
      inputs:
        deploymentScope: Resource Group
        azureResourceManagerConnection: ${{ parameters.serviceConnection }}
        subscriptionId: $(subscriptionId)
        action: Create Or Update Resource Group
        resourceGroupName: $(azureResourceGroup)
        location: $(location)
        templateLocation: Linked artifact
        csmFile: 'infra/our-test-app/main.json' # created by bash script
        deploymentMode: Incremental
        deploymentOutputs: deployOutputs

    - task: PowerShell@2
      name: 'SetOutputVariables'
      displayName: "Set Output Variables"
      inputs:
        targetType: inline
        script: |
          $armOutputObj = '$(deployOutputs)' | ConvertFrom-Json
          $armOutputObj.PSObject.Properties | ForEach-Object {
            $keyname = $_.Name
            $value = $_.Value.value

            # Creates a standard pipeline variable
            Write-Output "##vso[task.setvariable variable=$keyName;]$value"

            # Creates an output variable
            Write-Output "##vso[task.setvariable variable=$keyName;issecret=true;isOutput=true]$value"

            # Display keys in pipeline
            Write-Output "output variable: $keyName"
          }
        pwsh: true

Above we can see:

• the Bicep get compiled to ARM
• the ARM is deployed to Azure, with deploymentOutputs being passed out at the end
• the outputs are turned into secret output variables inside the pipeline (the names of which are printed to the console)

With the above in place, we now have all of our variables in place; blobStorageConnectionString, blobContainerName, eventHubNamespaceConnectionString and eventHubName. These could now be consumed in whatever way is useful. Consider the following:

    - task: UseDotNet@2
      displayName: 'Install .NET Core SDK 3.1.x'
      inputs:
        packageType: 'sdk'
        version: 3.1.x

    - task: DotNetCoreCLI@2
      displayName: "dotnet run eventhub test"
      inputs:
        command: 'run'
        arguments: 'eventhub test --eventHubNamespaceConnectionString "$(eventHubNamespaceConnectionString)" --eventHubName "$(eventHubName)" --blobStorageConnectionString "$(blobStorageConnectionString)" --blobContainerName "$(blobContainerName)"'
        workingDirectory: '$(Build.SourcesDirectory)/OurTestApp'

Here we run a .NET application and pass it our connection strings. Please note, there's nothing .NET specific about what we're doing above - it could be any kind of application, bash script or similar that consumes our connection strings. The significant thing is that we can acquire connection strings in an automated fashion, for use in whichever manner pleases us.

Managed Identity, Azure SQL and Entity Framework

Managed Identity offers a very secure way for applications running in Azure to connect to Azure SQL databases. It's an approach that does not require code changes; merely configuration of connection string and associated resources. Hence it has a good developer experience. Importantly, it allows us to avoid exposing our database to username / password authentication, and hence making it a tougher target for bad actors.

This post talks us through using managed identity for connecting to Azure SQL.

Integrated Security=true

Everyone is deploying to the cloud. Few are the organisations that view deployment to data centers they manage as the future. This is generally a good thing, however in the excitement of the new, it's possible to forget some of the good properties that "on premise" deployment afforded when it came to connectivity and authentication.

I speak of course, of our old friend Integrated Security=true. When you seek to connect a web application to a database, you'll typically use some kind of database connection string. And back in the day, it may have looked something like this:

Data Source=myServer;Initial Catalog=myDB;Integrated Security=true;

The above provides a database server, a database and also Integrated Security=true. When you see Integrated Security=true, what you're essentially looking at is an instruction to use the identity that an application is running under (typically called a "service account") as the authentication credential to secure access to the database. Under the covers, this amounts to Windows Authentication.

The significant thing about this approach is that it is more secure than using usernames and passwords in the connection string. If you have to use username and password to authenticate, then you need to persist them somewhere - so you need to make sure that's secure. Also, if someone manages to acquire that username and password, they're free to get access to the database and do malicious things.

Bottom line: the less you are sharing authentication credentials, the better your security. Integrated Security is a harder nut to crack than username and password. The thing to note about the above phrase is "Windows Authentication". Web Apps in Azure / AWS etc do not typically use Windows Authentication when it comes to connecting to the database. Connecting with username / password is far more common.

What if there was a way to have the developer experience of Integrated Security=true without needing to use Windows Authentication? There is.

Managed Identity

The docs express the purpose of managed identity well:

A common challenge for developers is the management of secrets and credentials to secure communication between different services. On Azure, managed identities eliminate the need for developers having to manage credentials by providing an identity for the Azure resource in Azure AD and using it to obtain Azure Active Directory (Azure AD) tokens

Historically a certain amount of ceremony was required to use managed identity to connect to a database, and could involve augmenting a DbContext like so:

public MyDbContext(DbContextOptions options) : base(options) {
    var conn = (Microsoft.Data.SqlClient.SqlConnection)Database.GetDbConnection();
    var credential = new DefaultAzureCredential();
    var token = credential
        .GetToken(
            new Azure.Core.TokenRequestContext(new[] { "https://database.windows.net/.default" })
        );
    conn.AccessToken = token.Token;
}

This mechanism works, and has the tremendous upside of no longer requiring credentials be passed in a connection string. However, as you can see this isn't the simplest of setups. And also, what if you don't want to use managed identity when you're developing locally? This approach has baggage and forces us to make code changes.

Connection String alone

The wonderful aspect of the original Integrated Security=true approach, was that there were no code changes required; one need only supply the connection string. Just configuration.

This is now possible with Azure SQL thanks to this PR to the Microsoft.Data.SqlClient nuget package. (Incidentally, Microsoft.Data.SqlClient is the successor to System.Data.SqlClient.)

Support for connection string managed identities shipped with v2.1. Connection strings can look slightly different depending on the type of managed identity you're using:

// For System Assigned Managed Identity
"Server:{serverURL}; Authentication=Active Directory MSI; Initial Catalog={db};"

// For System Assigned Managed Identity
"Server:{serverURL}; Authentication=Active Directory Managed Identity; Initial Catalog={db};"

// For User Assigned Managed Identity
"Server:{serverURL}; Authentication=Active Directory MSI; User Id={ObjectIdOfManagedIdentity}; Initial Catalog={db};"

// For User Assigned Managed Identity
"Server:{serverURL}; Authentication=Active Directory Managed Identity; User Id={ObjectIdOfManagedIdentity}; Initial Catalog={db};"

Regardless of the approach, you can see that none of the connection strings have credentials in them. And that's special.

Usage with Entity Framework Core 5

If you're using Entity Framework Core, you might be struggling to get this working and encountering strange error messages. In my ASP.NET project I had a dependendency on Microsoft.EntityFrameworkCore.SqlServer@5.

If you look close above, you'll see that the package has a dependency on Microsoft.Data.SqlClient, but crucially on 2.0.1 or greater. So if dotnet has installed a version of Microsoft.Data.SqlClient which is less than 2.1 then the functionality required will not be installed. The resolution is simple, ensure that the required version is installed:

dotnet add package Microsoft.Data.SqlClient --version 2.1.2

The version which we want to use is 2.1 (or greater) and fortunately that is compatible with Entity Framework Core 5. Incidentally, when Entity Framework Core 6 ships it will no longer be necessary to manually specify this dependency as it already requires Microsoft.Data.SqlClient@2.1 as a minimum.

User Assigned Managed Identity

If you're using user assigned managed identity, you'll need to supply the object id of your managed identity, which you can find in the Azure Portal:

You can configure this in ARM as well, but cryptically, the object id goes by the nom de plume of principalId (thanks to my partner in crime John McCormick for puzzling that out):

"CONNECTIONSTRINGS__OURDBCONNECTION": "[concat('Server=tcp:', parameters('sqlServerName') , '.database.windows.net,1433;Initial Catalog=', parameters('sqlDatabaseName'),';Authentication=Active Directory MSI',';User Id=', reference(resourceId('Microsoft.ManagedIdentity/userAssignedIdentities/', parameters('managedIdentityName')), '2018-11-30').principalId)]"

That's it! With managed identity handling your authentication you can sleep easy, knowing you should be in a better place security wise.