Create a Pipeline with the Azure DevOps API

Creating an Azure Pipeline using the Azure DevOps REST API is possible, but badly documented. This post goes through how to do this.

curling a pipeline

The documentation for creating an Azure Pipeline using the Azure DevOps API is somewhat lacking. However it isn't actually too hard, you just need the recipe.

Here's a curl to make you a pipeline:

curl  --user '':'PERSONAL_ACCESS_TOKEN' --header "Content-Type: application/json" --header "Accept:application/json" https://dev.azure.com/organisation-name/sandbox/_apis/pipelines?api-version=6.1-preview.1 -d @makepipeline.json

Looking at the above there's two things you need:

1. A personal access token. You can make one of those here: https://dev.azure.com/organisation-name/_usersSettings/tokens (where organisation-name is the name of your organisation)
2. A makepipeline.json file, which contains the details of the pipeline you want to create:

{
  "folder": null,
  "name": "pipeline-made-by-api",
  "configuration": {
    "type": "yaml",
    "path": "/azure-pipelines.yml",
    "repository": {
      "id": "guid-of-repo-id",
      "name": "my-repo",
      "type": "azureReposGit"
    }
  }
}

Let's talk through the significant properties above:

• folder - can be null if you'd like the pipeline to be created in the root of Pipelines; otherwise provide the folder name. Incidentally a null will be translated into a value of \\ which appears to be the magic value which represents the root.
• name - your pipeline needs a name
• path - this is the path to the yaml pipelines file in the repo. Note we're creating the pipeline itself here; what's actually in the pipeline sits in that file.
• repository.id - this is the guid that represents the repo you're creating the pipeline for. You can find this out by going to your equivalent https://dev.azure.com/organisation-name/project-name/_settings/repositories (substituting in appropriate values) and looking up your repository there.
• repository.name - the name of your repo

When you execute your curl you should be returned some JSON along these lines:

{
  "_links": {
    "self": {
      "href": "https://dev.azure.com/organisation-name/2184049d-8bc4-484a-91e6-00fca6b5b19f/_apis/pipelines/975?revision=1"
    },
    "web": {
      "href": "https://dev.azure.com/organisation-name/2184049d-8bc4-484a-91e6-00fca6b5b19f/_build/definition?definitionId=975"
    }
  },
  "configuration": {
    "path": "/azure-pipelines.yml",
    "repository": {
      "id": "9a72560d-1622-4016-93dd-32ac85b96d03",
      "type": "azureReposGit"
    },
    "type": "yaml"
  },
  "url": "https://dev.azure.com/organisation-name/2184049d-8bc4-484a-91e6-00fca6b5b19f/_apis/pipelines/975?revision=1",
  "id": 975,
  "revision": 1,
  "name": "pipeline-made-by-api",
  "folder": "\\"
}

And inside Azure DevOps you'll now have a shiny new pipeline:

Azure DevOps Node API: The missing episodes

I've been taking a good look at the REST API for Azure DevOps. I'm delighted to say that it's a very full API. However, there's quirks.

I'm writing a tool that interrogates Azure DevOps in order that it can construct release documentation. That release documentation we would like to publish to the project wiki.

To make integration with Azure DevOps even easier, the ADO team have put a good amount of work into client libraries that allow you to code in your language of choice. In my case I'm writing a Node.js tool (using TypeScript) and happily the client lib for Node is written and published with TypeScript too. Tremendous! However, there is a "but" coming....

Wiki got a big ol' "but"

As I've been using the Node client lib, I've found minor quirks. Such as the GitApi.getRefs missing the pagination parts of the API.

Whilst the GitApi was missing some parameters on a method, the WikiApi was missing whole endpoints, such as the Pages - Create Or Update one. The various client libraries are auto-generated which makes contribution a difficult game. The lovely Matt Cooper has alerted the team

These clients are generated from the server-side controllers, and at a glance, I don't understand why those two parameters weren't included. Full transparency, we don't dedicate a lot of cycles here, but I will get it on the team's radar to investigate/improve.

In the meantime, I still had a tool to write.

Handrolled Wiki API

Whilst the Node.js client lib was missing some crucial pieces, there did seem to be a way forward. Using the API directly; not using the client lib to do our HTTP and using axios instead. Happily the types we needed were still available for be leveraged.

Looking at the docs it seemed it ought to be simple:

https://docs.microsoft.com/en-us/rest/api/azure/devops/?view=azure-devops-rest-6.1#assemble-the-request

But when I attempted this I found my requests erroring out with 203 Non-Authoritative Informations. It didn't make sense. I couldn't get a single request to be successful, they all failed. It occurred to me that the answer was hiding in node_modules. I'd managed to make successful requests to the API using the client lib. What was it doing that I wasn't?

The answer ended up being an authorization one-liner:

    const request = await axios({
        url,
        headers: {
            Accept: 'application/json',
            'Content-Type': 'application/json',
            // This!
            Authorization: `Basic ${Buffer.from(`PAT:${adoPersonalAccessToken}`).toString('base64')}`,
            'X-TFS-FedAuthRedirect': 'Suppress',
        },
    });
}

With this in hand everything started to work and I found myself able to write my own clients to fill in the missing pieces from the client lib:

import axios from 'axios';
import { WikiPage, WikiPageCreateOrUpdateParameters, WikiType } from 'azure-devops-node-api/interfaces/WikiInterfaces';
import { IWikiApi } from 'azure-devops-node-api/WikiApi';

async function getWikiPage({
    adoUrl,
    adoProject,
    adoPat,
    wikiId,
    path,
}: {
    adoUrl: string;
    adoProject: string;
    adoPat: string;
    wikiId: string;
    path: string;
}) {
    try {
        const url = `${makeBaseApiUrl({
            adoUrl,
            adoProject,
        })}/wiki/wikis/${wikiId}/pages?${apiVersion}&path=${path}&includeContent=True&recursionLevel=full`;
        const request = await axios({
            url,
            headers: makeHeaders(adoPat),
        });

        const page: WikiPage = request.data;
        return page;
    } catch (error) {
        return undefined;
    }
}

async function createWikiPage({
    adoUrl,
    adoProject,
    adoPat,
    wikiId,
    path,
    data,
}: {
    adoUrl: string;
    adoProject: string;
    adoPat: string;
    wikiId: string;
    path: string;
    data: WikiPageCreateOrUpdateParameters;
}) {
    try {
        const url = `${makeBaseApiUrl({
            adoUrl,
            adoProject,
        })}/wiki/wikis/${wikiId}/pages?${apiVersion}&path=${path}`;

        const request = await axios({
            method: 'PUT',
            url,
            headers: makeHeaders(adoPat),
            data,
        });

        const newPage: WikiPage = request.data;
        return newPage;
    } catch (error) {
        return undefined;
    }
}

const apiVersion = "api-version=6.0";

/**
* Create the headers necessary to ake Azure DevOps happy
* @param adoPat Personal Access Token from ADO
*/
function makeHeaders(adoPat: string) {
    return {
        Accept: 'application/json',
        'Content-Type': 'application/json',
        Authorization: `Basic ${Buffer.from(`PAT:${adoPat}`).toString('base64')}`,
        'X-TFS-FedAuthRedirect': 'Suppress',
    };
}

/**
* eg https://dev.azure.com/{organization}/{project}/_apis
*/
function makeBaseApiUrl({ adoUrl, adoProject }: { adoUrl: string; adoProject: string }) {
    return `${adoUrl}/${adoProject}/_apis`;
}

With this I was able to write code like this:

    let topLevelPage = await getWikiPage({
        adoUrl,
        adoProject,
        adoPat,
        wikiId,
        path: config.wikiTopLevelName,
    });

    if (!topLevelPage)
        topLevelPage = await createWikiPage({
            adoUrl,
            adoProject,
            adoPat,
            wikiId,
            path: config.wikiTopLevelName,
            data: { content: '' },
        });

and the wikis were ours!