Pipeline Templates: Templates for Downloading and Caching

Earlier this year I published a pre-release version of an Azure DevOps extension that wrapped the ApkTool for extracting and re-packing Android APK files. Having spent more time working with Azure Pipelines, I realised that most of what I was doing with the extension could actually be done with one or more templates. Today I … Continue reading “Pipeline Templates: Templates for Downloading and Caching”

Earlier this year I published a pre-release version of an Azure DevOps extension that wrapped the ApkTool for extracting and re-packing Android APK files. Having spent more time working with Azure Pipelines, I realised that most of what I was doing with the extension could actually be done with one or more templates. Today I started the process of creating the templates… and unfortunately got side tracked in over engineering a couple of the templates. The upshot is that the Pipeline Templates library now has the following new templates (there’s no release yet but you can reference them off the master branch if you want to use them today).

  • Download-Url-To-File – As the name suggests this template can be used to download a url (source_url) to a specified file (target_path). The target_path parameter can be an absolute or relative path, and you can specify the overwrite_existing parameter if you want to force the url to be downloaded even if the file already exists.
  • Check-File-Exists – Again, the name says what’s in the box – this template checks to see if the specified file (path_to_check) exists. The response can either be an output variable (specified using the file_exists_variable parameter) or an error can be thrown (which will break the current build)
  • Download-And-Cache – Rather than having to download files every time a build is run, the files can be cached and reloaded for each build. This template makes this easy.
  • ApkTool-Install – This downloads and caches the two files required to run the apktool (apktool.jar and apktool.bat) on Windows.

Once I get to the point where I’m able to replicate the functionality of the Apktool extension I’ll publish these updates as a new release. In meantime, if there are any other templates you’d like to see, feel free to create an issue on the Pipeline Templates GitHub repo, or drop me a comment

Pipeline Templates: How to use a file for release notes?

When deploying a release to AppCenter you can specify release notes that get presented to the user when they go to download a new release. This week a question was asked as to how to specify release notes from a file when submitting a new app version to AppCenter. If you looked at the complete … Continue reading “Pipeline Templates: How to use a file for release notes?”

When deploying a release to AppCenter you can specify release notes that get presented to the user when they go to download a new release. This week a question was asked as to how to specify release notes from a file when submitting a new app version to AppCenter.

If you looked at the complete example I provided for building and deploying a Uno app to Android, iOS and Windows, you might be wondering how you can provide release notes at all since we didn’t specify any release notes when we used the AppCenter deploy template. However, if you take a look at the parameters list you can see that there is an appcenter_release_notes parameter which has a default value set. Use this parameter if you want to specify the release notes inline when invoking the template.

In the v0.5.1 release we added two new parameters that allow you to specify a file for release notes to be taken from: appcenter_release_notes_option and appcenter_release_notes_file. To specify a file for release notes, you first need to set the appcenter_release_notes_option parameter to file. Then you need to use the appcenter_release_notes_file parameter to specify the file that you want to use.

In theory this sounds easy: you add a release notes file to your repository and then simply provide a relative path to the file in the appcenter_release_notes_file parameter. This you will find does not work!!! The AppCenter deploy template does not know anything about your source code repository, since it’s designed to deploy the artifacts from a prior build stage.

Ok, so then the question is, how do we make the release notes file available to the AppCenter template? Well we pretty much answered that in the previous paragraph – you need to deploy it as an artifact from the build process.

Assuming you’re using one of the build templates from https://pipelinetemplates.com, you can easily do this by adding additional steps as part of the prePublish extension point. Here’s an example:

- template:  azure/mobile/[email protected]_templates
  parameters:
    # Stage name and whether it's enabled
    stage_name: 'Build_iOS' 
    build_ios_enabled: $(ios_enabled)
    # Version information
    full_version_number: '$(version_prefix).$(Build.BuildId)'
    # Solution to build
    solution_filename: $(solution_file)
    solution_build_configuration: $(solution_build_config)
    # Signing information
    ios_plist_filename: 'src/Apps/DotNet/Uno/InspectorUno/InspectorUno/InspectorUno.iOS/Info.plist'
    ios_cert_password: '$(ios_signing_certificate_password)'
    ios_cert_securefiles_filename: '$(ios_signing_certificate_securefiles_filename)'
    ios_provisioning_profile_securefiles_filename: '$(ios_provisioning_profile_securefiles_filename)'
    # Output information
    artifact_folder: $(artifact_ios_folder)
    application_package: $(ios_application_package)
    prePublish:
      - task: [email protected]2
        displayName: 'Copying release notes'
        inputs:
          contents: 'src/Apps/DotNet/releasenotes.txt'
          targetFolder: '$(build.artifactStagingDirectory)/$(artifact_ios_folder)'
          flattenFolders: true
          overWrite: true

- template:  azure/mobile/[email protected]_templates
  parameters:
    # Stage name and dependencies
    stage_name: 'Deploy_iOS'
    depends_on: 'Build_iOS'
    deploy_appcenter_enabled: $(ios_enabled)
    environment_name: $(appcenter_environment)
    # Build artifacts
    artifact_folder: $(artifact_ios_folder)
    application_package: $(ios_application_package)
    # Deployment to AppCenter
    appcenter_service_connection: $(appcenter_service_connection)
    appcenter_organisation: $(appcenter_organisation)
    appcenter_applicationid: $(appcenter_ios_appid)
    appcenter_distribution_group_ids: '5174f212-ea8c-4df1-b159-391200d7af5f'
    appcenter_release_notes_option: file
    appcenter_release_notes_file: 'releasenotes.txt'

Note that the CopyFiles task copies the release notes into the artifact_folder sub-folder of the staging directory. This is important because the AppCenter deploy template will look in the sub-folder for the release notes file specified using the appcenter_release_notes_file parameter.

Pipeline Templates: Stage Dependency Fix and Improved Docs

We just released v0.5.1 of the Pipeline Templates – templates for creating build pipelines for Azure DevOps. This was a minor increment but fixed an issue that emerged due to a change in the validation of pipelines by Azure DevOps. Before we get into the details of what is included in v0.5.1, the other big … Continue reading “Pipeline Templates: Stage Dependency Fix and Improved Docs”

We just released v0.5.1 of the Pipeline Templates – templates for creating build pipelines for Azure DevOps. This was a minor increment but fixed an issue that emerged due to a change in the validation of pipelines by Azure DevOps.

Before we get into the details of what is included in v0.5.1, the other big news is that Pipeline Templates has a new documentation site.

http://pipelinetemplates.com

This site uses GitHub Pages and as such is generated from the markdown files in the project itself. Currently the site has a getting started guide, followed by a summary of each of the templates. Hopefully over time this can grow as we include more examples and extend the list of templates.

Pipeline Templates v0.5.1

Here’s a summary of what’s in this release:

Breaking Changes:

  • None

Other Changes:

  • Fix: build-xamarin-[iOS/android/windows].yml and deploy-appcenter.yml – changed depends_on parameter from stageList to string to fix validation issue introduced by Azure DevOps
  • Added NuGetInstallAndRestore.yml based on a template suggest by Damien Aicheh in his post on how to Add nightly builds to your Xamarin applications using Azure DevOps
  • Updated deploy-appcenter.yml to include additional parameters for controlling how apps are deployed via AppCenter. For example you can now specify release notes as a file, and you can flag whether a release is a mandatory update or not.
  • Added Markdown files that document each of the templates. Go to https://pipelinetemplates.com to see the rendered output.

Pipeline Templates: Complete Azure Pipelines Example for a Uno Project for iOS, Android and Windows

My last post was a bit of a long one as it covered a bunch of steps for setting up the bits and pieces required for signing an application for different platforms. In this post I just wanted to provide a complete example that shows a single multi-stage (6 in total) Azure Pipelines pipeline for … Continue reading “Pipeline Templates: Complete Azure Pipelines Example for a Uno Project for iOS, Android and Windows”

My last post was a bit of a long one as it covered a bunch of steps for setting up the bits and pieces required for signing an application for different platforms. In this post I just wanted to provide a complete example that shows a single multi-stage (6 in total) Azure Pipelines pipeline for building a Uno application for iOS, Android and Windows (UWP) and releasing them to App Center.

Secure Files

In my last post I showed how to create and populate Secure Files in Azure Pipelines. Any certificate or provisioning profile you need to use in your pipeline should be added to the Secure Files section of the Library in Azure Pipeline. My list of Secure Files looks like this:

Here we can see that we have the signing certificates for iOS and Windows, and the keystore for Android. Then we have two iOS provisioning profiles, one for my XF application and the other for my Uno application.

Variable Groups

I’ve extracted most of the variables I use in my pipeline into one of two variable groups:

The Common Build Variables are those variables that can be reused across multiple projects.

The Inspector Uno Build Variables are those variables that are specific to this project. For example it includes the AppCenter ids for the iOS, Android and Windows applications. It also includes the iOS provisioning profile which is specifically tied to this application.

Pipeline

Here’s the entire pipeline:

resources:
  repositories:
    - repository: builttoroam_templates
      type: github
      name: builttoroam/pipeline_templates
      ref: refs/tags/v0.5.0
      endpoint: github_connection
  
variables:
  - group: 'Common Build Variables'
  - group: 'Inspector Uno Build Variables'
  - name: ios_enabled
    value: 'true'
  - name: windows_enabled
    value: 'true'
  - name: android_enabled
    value: 'true'
 
stages:
- template:  azure/mobile/[email protected]_templates
  parameters:
    # Stage name and whether it's enabled
    stage_name: 'Build_Android'
    build_android_enabled: $(android_enabled)
    # Version information
    full_version_number: '$(version_prefix).$(Build.BuildId)'
    # Signing information
    secure_file_keystore_filename: '$(android_keystore_filename)'
    keystore_alias: '$(android_keystore_alias)'
    keystore_password: '$(android_keystore_password)'
    # Solution to build
    solution_filename: $(solution_file)
    solution_build_configuration: $(solution_build_config)
    # Output information
    artifact_folder: $(artifact_android_folder)
    application_package: $(android_application_package)

- template:  azure/mobile/[email protected]_templates
  parameters:
    # Stage name and dependencies
    stage_name: 'Deploy_Android'
    depends_on: 'Build_Android'
    deploy_appcenter_enabled: $(android_enabled)
    environment_name: $(appcenter_environment)
    # Build artifacts
    artifact_folder: $(artifact_android_folder)
    application_package: $(android_application_package)
    # Signing information (for Android repack to APK)
    secure_file_keystore_filename: '$(android_keystore_filename)'
    keystore_alias: '$(android_keystore_alias)'
    keystore_password: '$(android_keystore_password)'
    # Deployment to AppCenter
    appcenter_service_connection: $(appcenter_service_connection)
    appcenter_organisation: $(appcenter_organisation)
    appcenter_applicationid: $(appcenter_android_appid)


- template:  azure/mobile/[email protected]_templates
  parameters:
    # Stage name and whether it's enabled
    stage_name: 'Build_Windows'
    build_windows_enabled: $(windows_enabled)
    # Version information
    full_version_number: '$(version_prefix).$(Build.BuildId)'
    # Signing information
    windows_cert_securefiles_filename: '$(windows_signing_certificate_securefiles_filename)'
    windows_cert_password: '$(windows_signing_certificate_password)'
    # Solution to build
    solution_filename: $(solution_file)
    solution_build_configuration: $(solution_build_config)
    # Output information
    artifact_folder: $(artifact_windows_folder)
    application_package: $(windows_application_package)

- template:  azure/mobile/[email protected]_templates
  parameters:
    # Stage name and dependencies
    stage_name: 'Deploy_Windows'
    depends_on: 'Build_Windows'
    deploy_appcenter_enabled: $(windows_enabled)
    environment_name: $(appcenter_environment)
    # Build artifacts
    artifact_folder: $(artifact_windows_folder)
    application_package: $(windows_application_package)
    # Deployment to AppCenter
    appcenter_service_connection: $(appcenter_service_connection)
    appcenter_organisation: $(appcenter_organisation)
    appcenter_applicationid: $(appcenter_windows_appid)

- template:  azure/mobile/[email protected]_templates
  parameters:
    # Stage name and whether it's enabled
    stage_name: 'Build_iOS' 
    build_ios_enabled: $(ios_enabled)
    # Version information
    full_version_number: '$(version_prefix).$(Build.BuildId)'
    # Solution to build
    solution_filename: $(solution_file)
    solution_build_configuration: $(solution_build_config)
    # Signing information
    ios_plist_filename: 'src/Apps/DotNet/Uno/InspectorUno/InspectorUno/InspectorUno.iOS/Info.plist'
    ios_cert_password: '$(ios_signing_certificate_password)'
    ios_cert_securefiles_filename: '$(ios_signing_certificate_securefiles_filename)'
    ios_provisioning_profile_securefiles_filename: '$(ios_provisioning_profile_securefiles_filename)'
    # Output information
    artifact_folder: $(artifact_ios_folder)
    application_package: $(ios_application_package)

- template:  azure/mobile/[email protected]_templates
  parameters:
    # Stage name and dependencies
    stage_name: 'Deploy_iOS'
    depends_on: 'Build_iOS'
    deploy_appcenter_enabled: $(ios_enabled)
    environment_name: $(appcenter_environment)
    # Build artifacts
    artifact_folder: $(artifact_ios_folder)
    application_package: $(ios_application_package)
    # Deployment to AppCenter
    appcenter_service_connection: $(appcenter_service_connection)
    appcenter_organisation: $(appcenter_organisation)
    appcenter_applicationid: $(appcenter_ios_appid)


Pipeline Templates v0.5.0

Most of the v0.5.0 release has been tidying things up, reducing the number of required parameters by making the template smarter and increasing consistency across the templates

Breaking Changes:

  • build-xamarin-android.yml – changed build_platform to solution_target_platform parameter
  • build-xamarin-windows.yml – changed build_platform to solution_target_platform parameter
  • build-xamarin-windows.yml – changed windows_appxupload_name parameter to windows_upload_name to reflect support for msix

Other Changes:

  • build-xamarin-[iOS/android/windows].yml – added depends_on parameter so that the stages can be ordered
  • build-xamarin-[iOS/android/windows].yml – artifact_name, artifact_folder and application_package are no longer required and have default values
  • build-xamarin-android.yml – supports building either aab or apk based on the application_package parameter
  • build-xamarin-windows.yml – windows_upload_name parameter isn’t required as it has a default value based on the bundle name
  • deploy-appcenter.yml – added conditions to all steps so that pipeline doesn’t break if build stage doesn’t generate any output
  • All template – documentation added to parameters and steps

Deploy Xamarin.Forms Apps to App Center from a Azure Multi-Stage Pipeline using Templates and Environments that Require Manual Approval

Wow, that title’s a mouthful, and I didn’t add in there that I’ve just pushed v0.2.0 release of the Pipeline Templates repository. In this post we’re going to add stages to a YAML based Azure DevOps pipeline in order to deploy a Xamarin.Forms application to AppCenter for testing. We’ll also be using on the of … Continue reading “Deploy Xamarin.Forms Apps to App Center from a Azure Multi-Stage Pipeline using Templates and Environments that Require Manual Approval”

Wow, that title’s a mouthful, and I didn’t add in there that I’ve just pushed v0.2.0 release of the Pipeline Templates repository. In this post we’re going to add stages to a YAML based Azure DevOps pipeline in order to deploy a Xamarin.Forms application to AppCenter for testing. We’ll also be using on the of the latest features of the Azure DevOps YAML based pipelines, Environments, to insert a manual approval gate into our multi-stage pipeline.

Pipeline Templates v0.2.0

Before we get into talking about releasing apps to AppCenter I just wanted to reiterate that there is a new release of the Pipeline Templates repository with the following changes:

  • Added a new parameter, stage_name, to iOS, Android and Windows build templates. It has a default value, which matches the value previously specified on the stage element in the template, so won’t break any existing builds. This parameter can be set by the calling pipeline so that the stage is given a known name, which can then be referenced by other stages in the dependsOn element.
  • Added deployappcenter.yml template that can be used to deploy iOS, Android and Windows apps to AppCenter. For Android, if the application_package parameter is an .aab file, the calling pipeline will also need to supply keystore information. AppCenter doesn’t support .aab files, so the pipeline uses bundletool to generate and sign a fat apk, which is submitted to AppCenter.

Deploy to AppCenter

With a classic pipelines in Azure DevOps, you can setup separate build and release pipelines. However, YAML pipelines don’t differentiate between build and release pipelines; instead you can split a single pipeline into multiple stages (as we demonstrated in the previous post when we used different templates to create different stages in the build process).

To deploy apps to AppCenter we could simply create a template, similar to what we did with the build templates, that includes the tasks necessary to deploy to AppCenter, and then add new stages to our pipeline for each app we want to deploy. However, this would limit our ability to take advantage of some of the deployment specific features that are available in Azure DevOps. For this reason, the AppCenter template makes use of a deployment job in order to do the steps necessary to release an app to AppCenter.

Using the AppCenter Template

The following YAML pipeline provides an example of using the new AppCenter deployment template to deploy Windows (UWP), Android and iOS applications to AppCenter. Note that the ref element for the repository resource has been updated to point to the v0.2.0 tag.

resources:
  repositories:
    - repository: builttoroam_templates
      type: github
      name: builttoroam/pipeline_templates
      ref: refs/tags/v0.2.0
      endpoint: github_connection
  
variables:
  - group: 'Inspector XF Build Variables'
  
stages:
## Build stages excluded for brevity
- template:  azure/mobile/[email protected]_templates
  parameters:
    stage_name: 'Deploy_Windows'
    depends_on: 'Build_Windows'
    environment_name: 'Inspector-Alpha'
    artifact_name: 'inspector-build'
    artifact_folder: 'Windows_output'
    application_package: 'Inspector-XF-Windows.appxbundle'
    appcenter_service_connection: 'AppCenterInspectorCI'
    appcenter_organisation: 'thenickrandolph'
    appcenter_applicationid: 'Inspector-XF-UWP'
    appcenter_release_notes: 'Release from deploy pipeline'

- template:  azure/mobile/[email protected]_templates
  parameters:
    stage_name: 'Deploy_Android'
    depends_on: 'Build_Android'
    environment_name: 'Inspector-Alpha'
    artifact_name: 'inspector-build'
    artifact_folder: 'Android_output'
    application_package: 'Inspector-XF-Android.aab'
    appcenter_service_connection: 'AppCenterInspectorCI'
    appcenter_organisation: 'thenickrandolph'
    appcenter_applicationid: 'Inspector-XF-Android-Alpha'
    appcenter_release_notes: 'Release from deploy pipeline'
    secure_file_keystore_filename: '$(android_keystore_filename)'
    keystore_alias: '$(android_keystore_alias)'
    keystore_password: '$(android_keystore_password)'

- template:  azure/mobile/[email protected]_templates
  parameters:
    stage_name: 'Deploy_iOS'
    depends_on: 'Build_iOS'
    environment_name: 'Inspector-Alpha'
    artifact_name: 'inspector-build'
    artifact_folder: 'iOS_output'
    application_package: 'Inspector-XF-iOS.ipa'
    appcenter_service_connection: 'AppCenterInspectorCI'
    appcenter_organisation: 'thenickrandolph'
    appcenter_applicationid: 'Inspector-XF-iOS-Alpha'
    appcenter_release_notes: 'Release from deploy pipeline'

There are a couple of prerequisites that need to be setup in order for the deploy stages to work:

  • Each deploy stage specifies a depends_on property. This needs to correlate to the stage_name property specified on the corresponding build stage.
  • The artifact name, artifact folder and application_package properties need to match to the values used in the corresponding build stage.
  • A Service Connection needs to be established between Azure DevOps and AppCenter (in this case it was given the name ‘AppCenterInspectorCI’)
  • An application needs to be registered in AppCenter for each target platform. Each deploy stage needs the organisation (ie username or org_name) and applicationid (app_identifier). See the documentation on the AppCenterDistribute task for more information on how to find these values for your AppCenter apps.

Manual Approval with Environments

I mentioned earlier that using a deployment job would allow us to take advantage of deployment specific features. This is an area that’s currently under development and we’d expect to see more features lighting up in this area over time.

One feature that’s available today is the ability to add manual approval requirement to a deployment job. However, unlike in the classic pipeline where you’d create a manual approval requirement directly on the release pipeline, on a YAML pipeline you actually need to associated the deployment job with an environment and then add a manual approval on the environment.

You may have noticed that each of the deploy stages in the example YAML specified the environment_name property. This defines which environment you’re going to be deploying to. At this stage the only thing you can use this for in terms of deploying a mobile application is to require manual approval for the stage to continue. Let’s step through creating the environment and the approval requirement and you’ll see what I mean.

Create an Azure Pipelines Environment

Under the Pipelines tab, select Environments and then click the Create environment button in the center of the screen.

Next, provide a name for your environment and click Create. At this stage we don’t need to define any resources, so you can leave the default selection of “None”. The name that you specify for your environment has to match what you use as the environment_name property on the template.

Adding Manual Approval to the Environment

In order to add a manual approval requirement to an environment, simply open the environment (if you’ve just created the environment you’re already there). From the drop-down menu in the top-right corner, select Approvals and checks.

Next, click the + button in the top right corner.

Select Approvals, and then click Next

In the Approvals flyout you can specify a list of users and/or groups that need to approve a release to the environment. If you specify multiple users, each user needs to approve the release. If you specify a group, only one person in the group needs to approve the release.

In the Approvals flyout you can also specify a timeout; if the deployment isn’t approved for an environment within the timeout, the pipeline will fail.

Note: Currently there are no emails, or other notifications, sent to approvers. If you limit the timeout, once the specified period has elapsed if the environment hasn’t been approved, the pipeline fail and a notification will be sent out. The stage in the pipeline that failed can then be manually run and again, approval for the environment will be required.

Running the Build and Deploy Pipeline

Now when we run the pipeline, what we see in the portal is three different rows (one for each platform) with two stages (a build and deploy stage).

We’ve set the dependsOn element on each of the build stages to an empty array (ie []) meaning that they have no dependencies (btw the default is that stages will be done in the sequence that they appear in the YAML file, unless you indicate there should be no dependencies). Depending on how many build agents you have at your disposal the build stages may run in sequence, or in parallel.

Eventually, when each of the build stages completes, the corresponding deploy stage will light up indicating that it’s waiting for a check to be passed. There’s also a message box inserted into the interface to draw attention to the required approval.

Once all three of the build stages are complete, there are 3 approvals required; one for each of the deploy stages. The way we’ve structured the pipeline you have to approve the deployment for each platform.

If you wanted to require only a single approval for all three platforms you could inject an additional stage that was dependent on all three build stages. The approval would be required for this single stage, and then each of the subsequent deploy stages would be permitted to continue without further checks. The downside of this would be that you would have to wait for all builds to fail before you could deploy the app to any platform.

Summary

In this post we’ve looked at using a pipeline template for deploying Xamarin.Forms applications to AppCenter for testing across Windows (UWP), iOS and Android. In actual fact, and what I didn’t point out earlier, the deploy template can be used for any iOS, Android or Windows (UWP) app, not just a Xamarin.Forms application.

I also walked through setting up an environment so that you could add a manual approval step to the deployment process. Whilst the YAML pipelines don’t yet have all the features of the classic release pipeline, you can see from the way that the components connect and the UI that’s built in the portal that the foundations have been laid for future features to be built on.