Change supporting infrastructure to use Azure Virtual Machine Scale Sets for vcpkg's PR builds, which should both improve our PR build times and reduce Azure spending by shutting down machines when they aren't being used.

Included is a script that sets up all vcpkg's Azure infrastructure for Windows PR tests, and several updates to baselines. The baseline updates are generally caused by an updated copy of the MSVC++ compiler caused by updating the VMs, but some are caused by missed failures only detected now because this did a cleared out archives directory first.

Some of the build infrastructure isn't what I'd call 'pretty' (e.g. we're split into more scripts and such than I'd like) but this mirrors how our existing PR system works.

It is expected that the existing vcpkg Windows PR system will hate these baseline updates so we'll need to merge this, then remove that (duplicate) workflow immediately afterwards, then delete all the Windows VMs powering the old infrastructure.

scripts/azure-pipelines/analyze-test-results.ps1

@@ -0,0 +1,778 @@
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: MIT
+#
+
+<#
+.SYNOPSIS
+Analyze the test results as output by the CI system.
+
+.DESCRIPTION
+Takes the set of port test results from $logDir,
+and the baseline from $baselineFile, and makes certain that the set
+of failures we expected are exactly the set of failures we got.
+Then, uploads the logs from any unexpected failures.
+
+.PARAMETER logDir
+Directory of xml test logs to analyze.
+
+.PARAMETER failurelogDir
+Path to the failure logs that need to be published to azure for inspection.
+
+.PARAMETER outputDir
+Where to write out the results of the analysis.
+
+.PARAMETER allResults
+Include tests that have no change from the baseline in the output.
+
+.PARAMETER errorOnRegression
+Output an error on test regressions.
+This will give a clean message in the build pipeline.
+
+.PARAMETER noTable
+Don't create or upload the markdown table of results
+
+.PARAMETER triplets
+A list of triplets to analyze; defaults to all triplets.
+
+.PARAMETER baselineFile
+The path to the ci.baseline.txt file in the vcpkg repository.
+#>
+[CmdletBinding()]
+Param(
+    [Parameter(Mandatory = $true)]
+    [string]$logDir,
+    [Parameter(Mandatory = $true)]
+    [string]$failurelogDir,
+    [Parameter(Mandatory = $true)]
+    [string]$outputDir,
+    [switch]$allResults,
+    [switch]$errorOnRegression,
+    [switch]$noTable,
+    [string[]]$triplets = @(),
+    [Parameter(Mandatory = $true)]
+    [string]$baselineFile
+)
+
+$ErrorActionPreference = 'Stop'
+
+if ( -not (Test-Path $logDir) ) {
+    [System.Console]::Error.WriteLine("Log directory does not exist: $logDir")
+    exit
+}
+if ( -not (Test-Path $outputDir) ) {
+    [System.Console]::Error.WriteLine("output directory does not exist: $outputDir")
+    exit
+}
+
+if ( $triplets.Count -eq 0 ) {
+    $triplets = @(
+        "x64-linux",
+        "x64-osx",
+        "arm-uwp",
+        "arm64-windows",
+        "x64-osx",
+        "x64-uwp",
+        "x64-windows-static",
+        "x64-windows",
+        "x86-windows"
+    )
+}
+
+
+<#
+.SYNOPSIS
+Creates an object the represents the test run.
+
+.DESCRIPTION
+build_test_results takes an XML file of results from the CI run,
+and constructs an object based on that XML file for further
+processing.
+
+.OUTPUTS
+An object with the following elements:
+    assemblyName:
+    assemblyStartDate:
+    assemblyStartTime:
+    assemblyTime:
+    collectionName:
+    collectionTime:
+    allTests: A hashtable with an entry for each port tested
+        The key is the name of the port
+        The value is an object with the following elements:
+            name: Name of the port (Does not include the triplet name)
+            result: Pass/Fail/Skip result from xunit
+            time: Test time in seconds
+            originalResult: Result as defined by Build.h in vcpkg source code
+            abi_tag: The port hash
+            features: The features installed
+
+.PARAMETER xmlFilename
+The path to the XML file to parse.
+#>
+function build_test_results {
+    [CmdletBinding()]
+    Param
+    (
+        [string]$xmlFilename
+    )
+    if ( ($xmlFilename.Length -eq 0) -or ( -not( Test-Path $xmlFilename))) {
+        #write-error "Missing file: $xmlFilename"
+        return $null
+    }
+
+    Write-Verbose "building test hash for $xmlFilename"
+
+    [xml]$xmlContents = Get-Content $xmlFilename
+
+    # This currently only supports one collection per assembly, which is the way
+    # the vcpkg tests are designed to run in the pipeline.
+    $xmlAssembly = $xmlContents.assemblies.assembly
+    $assemblyName = $xmlAssembly.name
+    $assemblyStartDate = $xmlAssembly."run-date"
+    $assemblyStartTime = $xmlAssembly."run-time"
+    $assemblyTime = $xmlAssembly.time
+    $xmlCollection = $xmlAssembly.collection
+    $collectionName = $xmlCollection.name
+    $collectionTime = $xmlCollection.time
+
+    $allTestResults = @{ }
+    foreach ( $test in $xmlCollection.test) {
+        $name = ($test.name -replace ":.*$")
+
+        # Reconstruct the original BuildResult enumeration (defined in Build.h)
+        #   failure.message - why the test failed (valid only on test failure)
+        #   reason - why the test was skipped (valid only when the test is skipped)
+        #    case BuildResult::POST_BUILD_CHECKS_FAILED:
+        #    case BuildResult::FILE_CONFLICTS:
+        #    case BuildResult::BUILD_FAILED:
+        #    case BuildResult::EXCLUDED:
+        #    case BuildResult::CASCADED_DUE_TO_MISSING_DEPENDENCIES:
+        $originalResult = "NULLVALUE"
+        switch ($test.result) {
+            "Skip" {
+                $originalResult = $test.reason.InnerText
+            }
+            "Fail" {
+                $originalResult = $test.failure.message.InnerText
+            }
+            "Pass" {
+                $originalResult = "SUCCEEDED"
+            }
+        }
+
+        $abi_tag = ""
+        $features = ""
+        foreach ( $trait in $test.traits.trait) {
+            switch ( $trait.name ) {
+                "abi_tag" { $abi_tag = $trait.value }
+                "features" { $features = $trait.value }
+            }
+        }
+
+        # If additional fields get saved in the XML, then they should be added to this hash
+        # also consider using a PSCustomObject here instead of a hash
+        $testHash = @{ name = $name; result = $test.result; time = $test.time; originalResult = $originalResult; abi_tag = $abi_tag; features = $features }
+        $allTestResults[$name] = $testHash
+    }
+
+    return @{
+        assemblyName      = $assemblyName;
+        assemblyStartDate = $assemblyStartDate;
+        assemblyStartTime = $assemblyStartTime;
+        assemblyTime      = $assemblyTime;
+        collectionName    = $collectionName;
+        collectionTime    = $collectionTime;
+        allTests          = $allTestResults
+    }
+}
+
+<#
+.SYNOPSIS
+Creates an object that represents the baseline expectations.
+
+.DESCRIPTION
+build_baseline_results converts the baseline file to an object representing
+the expectations set up by the baseline file. It records four states:
+    1) fail
+    2) skip
+    3) ignore
+    4) pass -- this is represented by not being recorded
+In other words, if a port is not contained in the object returned by this
+cmdlet, expect it to pass.
+
+.OUTPUTS
+An object containing the following fields:
+    collectionName: the triplet
+    fail: ports marked as fail
+    skip: ports marked as skipped
+    ignore: ports marked as ignore
+
+.PARAMETER baselineFile
+The path to vcpkg's ci.baseline.txt.
+
+.PARAMETER triplet
+The triplet to create the result object for.
+#>
+function build_baseline_results {
+    [CmdletBinding()]
+    Param(
+        $baselineFile,
+        $triplet
+    )
+    #read in the file, strip out comments and blank lines and spaces, leave only the current triplet
+    #remove comments, remove empty lines, remove whitespace, then keep only those lines for $triplet
+    $baseline_list_raw = Get-Content -Path $baselineFile `
+        | Where-Object { -not ($_ -match "\s*#") } `
+        | Where-Object { -not ( $_ -match "^\s*$") } `
+        | ForEach-Object { $_ -replace "\s" } `
+        | Where-Object { $_ -match ":$triplet=" }
+
+    #filter to skipped and trim the triplet
+    $skip_hash = @{ }
+    foreach ( $port in $baseline_list_raw | ? { $_ -match "=skip$" } | % { $_ -replace ":.*$" }) {
+        if ($skip_hash[$port] -ne $null) {
+            [System.Console]::Error.WriteLine("$($port):$($triplet) has multiple definitions in $baselineFile")
+        }
+        $skip_hash[$port] = $true
+    }
+    $fail_hash = @{ }
+    $baseline_list_raw | ? { $_ -match "=fail$" } | % { $_ -replace ":.*$" } | ? { $fail_hash[$_] = $true } | Out-Null
+    $ignore_hash = @{ }
+    $baseline_list_raw | ? { $_ -match "=ignore$" } | % { $_ -replace ":.*$" } | ? { $ignore_hash[$_] = $true } | Out-Null
+
+    return @{
+        collectionName = $triplet;
+        skip           = $skip_hash;
+        fail           = $fail_hash;
+        ignore         = $ignore_hash
+    }
+}
+
+<#
+.SYNOPSIS
+Analyzes the results of the current run against the baseline.
+
+.DESCRIPTION
+combine_results compares the results to the baselie, and generates the results
+for the CI -- whether it should pass or fail.
+
+.OUTPUTS
+An object containing the following:
+(Note that this is not the same data structure as build_test_results)
+    assemblyName:
+    assemblyStartDate:
+    assemblyStartTime:
+    assemblyTime:
+    collectionName:
+    collectionTime:
+    allTests: A hashtable of each port with a different status from the baseline
+        The key is the name of the port
+        The value is an object with the following data members:
+            name: The name of the port
+            result: xunit test result Pass/Fail/Skip
+            message: Human readable message describing the test result
+            time: time the current test results took to run.
+            baselineResult:
+            currentResult:
+            features:
+    ignored: list of ignored tests
+
+.PARAMETER baseline
+The baseline object to use from build_baseline_results.
+
+.PARAMETER current
+The results object to use from build_test_results.
+#>
+function combine_results {
+    [CmdletBinding()]
+    Param
+    (
+        $baseline,
+        $current
+    )
+
+    if ($baseline.collectionName -ne $current.collectionName) {
+        Write-Warning "Comparing mismatched collections $($baseline.collectionName) and $($current.collectionName)"
+    }
+
+    $currentTests = $current.allTests
+
+    # lookup table with the results of all of the tests
+    $allTestResults = @{ }
+
+    $ignoredList = @()
+
+    Write-Verbose "analyzing $($currentTests.count) tests"
+
+    foreach ($key in $currentTests.keys) {
+        Write-Verbose "analyzing $key"
+
+        $message = $null
+        $result = $null
+        $time = $null
+        $currentResult = $null
+        $features = $currentTest.features
+
+        $baselineResult = "Pass"
+        if ($baseline.fail[$key] -ne $null) {
+            Write-Verbose "$key is failing"
+            $baselineResult = "Fail"
+        }
+        elseif ( $baseline.skip[$key] -ne $null) {
+            Write-Verbose "$key is skipped"
+            $baselineResult = "Skip"
+        }
+        elseif ( $baseline.ignore[$key] -ne $null) {
+            $baselineResult = "ignore"
+        }
+
+        $currentTest = $currentTests[$key]
+
+        if ( $currentTest.result -eq $baselineResult) {
+            Write-Verbose "$key has no change from baseline"
+            $currentResult = $currentTest.result
+            if ($allResults) {
+                # Only marking regressions as failures but keep the skipped status
+                if ($currentResult -eq "Skip") {
+                    $result = "Skip"
+                }
+                else {
+                    $result = "Pass"
+                }
+                $message = "No change from baseline"
+                $time = $currentTest.time
+            }
+        }
+        elseif ( $baselineResult -eq "ignore") {
+            if ( $currentTest.result -eq "Fail" ) {
+                Write-Verbose "ignoring failure on $key"
+                $ignoredList += $key
+            }
+        }
+        else {
+            Write-Verbose "$key had a change from the baseline"
+
+            $currentResult = $currentTest.result
+            # Test exists in both test runs but does not match.  Determine if this is a regression
+            # Pass -> Fail = Fail (Regression)
+            # Pass -> Skip = Skip
+            # Fail -> Pass = Fail (need to update baseline)
+            # Fail -> Skip = Skip
+            # Skip -> Fail = Fail (Should not happen)
+            # Skip -> Pass = Fail (should not happen)
+
+            $lookupTable = @{
+                'Pass' = @{
+                    'Fail' = @('Fail', "Test passes in baseline but fails in current run. If expected update ci.baseline.txt with '$($key):$($current.collectionName)=fail'");
+                    'Skip' = @($null, 'Test was skipped due to missing dependencies')
+                };
+                'Fail' = @{
+                    'Pass' = @('Fail', "Test fails in baseline but now passes.  Update ci.baseline.txt with '$($key):$($current.collectionName)=pass'");
+                    'Skip' = @($null, 'Test fails in baseline but is skipped in current run')
+                };
+                'Skip' = @{
+                    'Fail' = @('Skip', "Test is skipped in baseline but fails in current run. Results are ignored")
+                    'Pass' = @('Skip', "Test is skipped in baseline but passes in current run. Results are ignored")
+                }
+            }
+            $resultList = $lookupTable[$baselineResult][$currentResult]
+            $result = $resultList[0]
+            $message = $resultList[1]
+            $time = $currentTest.time
+            Write-Verbose ">$key $message"
+        }
+
+        if ($result -ne $null) {
+            Write-Verbose "Adding $key to result list"
+            $allTestResults[$key] = @{ name = $key; result = $result; message = $message; time = $time; abi_tag = $currentTest.abi_tag; baselineResult = $baselineResult; currentResult = $currentResult; features = $features }
+        }
+    }
+
+    return @{
+        assemblyName      = $current.assemblyName;
+        assemblyStartDate = $current.assemblyStartDate;
+        assemblyStartTime = $current.assemblyStartTime;
+        assemblyTime      = $current.assemblyTime;
+        collectionName    = $current.collectionName;
+        collectionTime    = $current.collectionTime;
+        allTests          = $allTestResults;
+        ignored           = $ignoredList
+    }
+}
+
+<#
+.SYNOPSIS
+Takes the combined results object and writes it to an xml file.
+
+.DESCRIPTION
+write_xunit_results takes the results object from combine_results, and writes the
+results XML file to the correct location for the CI system to pick it up.
+
+.PARAMETER combined_results
+The results object from combine_results.
+#>
+function write_xunit_results {
+    [CmdletBinding()]
+    Param(
+        $combined_results
+    )
+    $allTests = $combined_results.allTests
+    $triplet = $combined_results.collectionName
+
+    $filePath = "$outputDir\$triplet.xml"
+    if (Test-Path $filePath) {
+        Write-Verbose "removing old file $filepath"
+        rm $filePath
+    }
+    Write-Verbose "output filename: $filepath"
+
+    $xmlWriter = New-Object System.Xml.XmlTextWriter($filePath, $Null)
+    $xmlWriter.Formatting = "Indented"
+    $xmlWriter.IndentChar = "`t"
+
+    $xmlWriter.WriteStartDocument()
+    $xmlWriter.WriteStartElement("assemblies")
+    $xmlWriter.WriteStartElement("assembly")
+    $xmlWriter.WriteAttributeString("name", $combined_results.assemblyName)
+    $xmlWriter.WriteAttributeString("run-date", $combined_results.assemblyStartDate)
+    $xmlWriter.WriteAttributeString("run-time", $combined_results.assemblyStartTime)
+    $xmlWriter.WriteAttributeString("time", $combined_results.assemblyTime)
+
+    $xmlWriter.WriteStartElement("collection")
+    $xmlWriter.WriteAttributeString("name", $triplet)
+    $xmlWriter.WriteAttributeString("time", $combined_results.collectionTime)
+
+    foreach ($testName in $allTests.Keys) {
+        $test = $allTests[$testName]
+
+        $xmlWriter.WriteStartElement("test")
+
+        $fullTestName = "$($testName):$triplet"
+        $xmlWriter.WriteAttributeString("name", $fullTestName)
+        $xmlWriter.WriteAttributeString("method", $fullTestName)
+        $xmlWriter.WriteAttributeString("time", $test.time)
+        $xmlWriter.WriteAttributeString("result", $test.result)
+
+        switch ($test.result) {
+            "Pass" { } # Do nothing
+            "Fail" {
+                $xmlWriter.WriteStartElement("failure")
+                $xmlWriter.WriteStartElement("message")
+                $xmlWriter.WriteCData($test.message)
+                $xmlWriter.WriteEndElement() #message
+                $xmlWriter.WriteEndElement() #failure
+            }
+            "Skip" {
+                $xmlWriter.WriteStartElement("reason")
+                $xmlWriter.WriteCData($test.message)
+                $xmlWriter.WriteEndElement() #reason
+            }
+        }
+
+        $xmlWriter.WriteEndElement() # test
+    }
+
+
+    $xmlWriter.WriteEndElement() # collection
+    $xmlWriter.WriteEndElement() # assembly
+    $xmlWriter.WriteEndElement() # assemblies
+    $xmlWriter.WriteEndDocument()
+    $xmlWriter.Flush()
+    $xmlWriter.Close()
+}
+
+<#
+.SYNOPSIS
+Saves the failure logs, and prints information to the screen for CI.
+
+.DESCRIPTION
+save_failure_logs takes the combined_results object, saves the failure
+logs to the correct location for the CI to pick them up, and writes pretty
+information to the screen for the CI logs, so that one knows what's wrong.
+
+.PARAMETER combined_results
+The results object from combine_results.
+#>
+function save_failure_logs {
+    [CmdletBinding()]
+    Param(
+        $combined_results
+    )
+    $triplet = $combined_results.collectionName
+    $allTests = $combined_results.allTests
+
+    # abi_tags of missing results (if any exist)
+    $missing_results = @()
+
+    foreach ($testName in $allTests.Keys) {
+        $test = $allTests[$testName]
+        if ($test.result -eq "Fail") {
+            $path_to_failure_Logs = Join-Path "$outputDir" "failureLogs"
+            if ( -not (Test-Path $path_to_failure_Logs)) {
+                mkdir $path_to_failure_Logs | Out-Null
+            }
+            $path_to_triplet_Logs = Join-Path $path_to_failure_Logs "$triplet"
+            if ( -not (Test-Path $path_to_triplet_Logs)) {
+                mkdir $path_to_triplet_Logs | Out-Null
+            }
+
+            $abi_tag = $test.abi_tag
+            $sourceDirectory = Join-Path "$failurelogDir" "$($abi_tag.substring(0,2))"
+            $sourceFilename = Join-Path $sourceDirectory "$abi_tag.zip"
+            Write-Verbose "searching for $sourceFilename"
+
+            if ( Test-Path $sourceFilename) {
+                Write-Verbose "found failure log file"
+
+                Write-Verbose "Uncompressing $sourceFilename to $outputDir\failureLogs\$triplet\"
+                Write-Host "Uncompressing $sourceFilename to $outputDir\failureLogs\$triplet\"
+
+                $destination = Join-Path (Join-Path "$outputDir" "failureLogs") "$triplet"
+
+                Expand-Archive -Path $sourceFilename -Destination $destination -Force
+            }
+            elseif ($test.currentState -eq "Pass") {
+                # The port is building, but is marked as expected to fail.  There are no failure logs.
+                # Write a log with instructions how to fix it.
+                Write-Verbose "The port is building but marked as expected to fail, adding readme.txt with fixit instructions"
+
+                $out_filename = Join-Path (Join-Path (Join-Path (Join-Path "$outputDir" "failureLogs") "$triplet") "$($test.name)") "readme.txt"
+
+                $message = "Congradulations! The port $($test.name) builds for $triplet!`n"
+                $message += "For the CI tests to recognize this, please update ci.baseline.txt in your PR.`n"
+                $message += "Remove the line that looks like this:`n"
+                $message += " $($test.name):$triplet=fail`n"
+                $message | Out-File $out_filename -Encoding ascii
+            }
+            else {
+                $missing_results += $test.abi_tag
+                Write-Verbose "Missing failure logs for $($test.name)"
+                Join-Path (Join-Path (Join-Path "$outputDir" "failureLogs") "$triplet" ) "$($test.name)" | % { mkdir $_ } | Out-Null
+            }
+
+
+
+            if ((Convert-Path "$outputDir\failureLogs\$triplet\$($test.name)" | Get-ChildItem).count -eq 0) {
+                Write-Verbose "The logs are empty, adding readme.txt"
+
+                $readme_path = Join-Path (Join-Path (Join-Path (Join-Path "$outputDir" "failureLogs") "$triplet") "$($test.name)") "readme.txt"
+
+                $message = "There are no build logs for $($test.name) build.`n"
+                $message += "This is usually because the build failed early and outside of a task that is logged.`n"
+                $message += "See the console output logs from vcpkg for more information on the failure.`n"
+                $message += "If the console output of the $($test.name) is missing you can trigger a rebuild`n"
+                $message += "in the test system by making any whitespace change to one of the files under`n"
+                $message += "the ports/$($test.name) directory or by asking a member of the vcpkg team to remove the`n"
+                $message += "tombstone for abi tag $abi_tag`n"
+                $message | Out-File $readme_path -Encoding ascii
+            }
+        }
+    }
+
+    if ($missing_results.count -ne 0) {
+        $missing_tag_filename = Join-Path (Join-Path (Join-Path "$outputDir" "failureLogs") "$triplet") "missing_abi_tags.txt"
+        $missing_results | Out-File -FilePath $missing_tag_filename -Encoding ascii
+    }
+    Write-Verbose "$triplet logs saved: $(Get-ChildItem $outputDir\failureLogs\$triplet\ -ErrorAction Ignore)"
+
+}
+
+<#
+.SYNOPSIS
+Writes a pretty summary table to the CI log.
+
+.DESCRIPTION
+Takes a hashtable which maps triplets to objects returned by the combine_results
+cmdlet, and a list of missing triplets, and prints a really pretty summary table
+to the CI logs.
+
+.PARAMETER complete_results
+A hashtable which maps triplets to combine_results objects.
+
+.PARAMETER missing_triplets
+A list of missing triplets.
+#>
+function write_summary_table {
+    [CmdletBinding()]
+    Param(
+        $complete_results,
+        $missing_triplets
+    )
+
+    $table = ""
+
+    foreach ($triplet in $complete_results.Keys) {
+        $triplet_results = $complete_results[$triplet]
+
+        if ($triplet_results.allTests.count -eq 0) {
+            $table += "$triplet CI build test results are clean`n`n"
+        }
+        else {
+            $portWidth = $triplet.length
+            #calculate the width of the first column
+            foreach ($testName in $triplet_results.allTests.Keys) {
+                $test = $triplet_results.allTests[$testName]
+                if ($portWidth -lt $test.name.length) {
+                    $portWidth = $test.name.length
+                }
+            }
+
+            # the header
+            $table += "|{0,-$portWidth}|result|features|notes`n" -f $triplet
+            $table += "|{0}|----|--------|-----`n" -f ("-" * $portWidth)
+
+            # add each port results
+            foreach ($testName in $triplet_results.allTests.Keys | Sort-Object) {
+                $test = $triplet_results.allTests[$testName]
+                $notes = ""
+                if ($test.result -eq 'Fail') {
+                    $notes = "**Regression**"
+                }
+                elseif ($test.result -eq 'Skip') {
+                    if ($test.currentResult -eq 'Fail') {
+                        $notes = "Previously skipped, not a regression"
+                    }
+                    else {
+                        $notes = "Missing port dependency"
+                    }
+                }
+                $notes = $test.message
+                $table += "|{0,-$portWidth}|{1,-4}|{2}|{3}`n" -f $test.name, $test.currentResult, $test.features, $notes
+            }
+            $table += "`n"
+        }
+        if ($triplet_results.ignored.Count -ne 0) {
+            $table += "The following build failures were ignored: $($triplet_results.ignored)`n"
+        }
+    }
+
+    # Add list of missing triplets to the table
+    foreach ($triplet in $missing_triplets.Keys) {
+        $table += "$triplet results are inconclusive because it is missing logs from test run`n`n"
+    }
+
+    $table
+}
+
+<#
+.SYNOPSIS
+Writes short errors to the CI logs.
+
+.DESCRIPTION
+write_errors_for_summary takes a hashtable from triplets to combine_results
+objects, and writes short errors to the CI logs.
+
+.PARAMETER complete_results
+A hashtable from triplets to combine_results objects.
+#>
+function write_errors_for_summary {
+    [CmdletBinding()]
+    Param(
+        $complete_results
+    )
+
+    $failure_found = $false
+
+    Write-Verbose "preparing error output for Azure Devops"
+
+    foreach ($triplet in $complete_results.Keys) {
+        $triplet_results = $complete_results[$triplet]
+
+        Write-Verbose "searching $triplet triplet"
+
+        # add each port results
+        foreach ($testName in $triplet_results.allTests.Keys) {
+            $test = $triplet_results.allTests[$testName]
+
+            Write-Verbose "checking $($testName):$triplet $($test.result)"
+
+            if ($test.result -eq 'Fail') {
+                $failure_found = $true
+                if ($test.currentResult -eq "pass") {
+                    [System.Console]::Error.WriteLine( `
+                            "PASSING, REMOVE FROM FAIL LIST: $($test.name):$triplet ($baselineFile)" `
+                    )
+                }
+                else {
+                    [System.Console]::Error.WriteLine( `
+                            "REGRESSION: $($test.name):$triplet. If expected, add $($test.name):$triplet=fail to $baselineFile." `
+                    )
+                }
+            }
+        }
+    }
+}
+
+
+$complete_results = @{ }
+$missing_triplets = @{ }
+foreach ( $triplet in $triplets) {
+    Write-Verbose "looking for $triplet logs"
+
+    # The standard name for logs is:
+    #   <triplet>.xml
+    # for example:
+    #   x64-linux.xml
+
+    $current_test_hash = build_test_results( Convert-Path "$logDir\$($triplet).xml" )
+    $baseline_results = build_baseline_results -baselineFile $baselineFile -triplet $triplet
+
+    if ($current_test_hash -eq $null) {
+        [System.Console]::Error.WriteLine("Missing $triplet test results in current test run")
+        $missing_triplets[$triplet] = "test"
+    }
+    else {
+        Write-Verbose "combining results..."
+        $complete_results[$triplet] = combine_results -baseline $baseline_results -current $current_test_hash
+    }
+}
+
+Write-Verbose "done analizing results"
+
+# If there is only one triplet, add the triplet name to the result table file
+if ($triplets.Count -eq 1) {
+    $result_table_name = $triplets[0]
+}
+else {
+    $result_table_name = ""
+}
+
+if (-not $noTable) {
+    $table_path = Join-Path "$outputDir" "result_table$result_table_name.md"
+
+    write_summary_table -complete_results $complete_results -missing_triplets $missing_triplets | Out-File -FilePath $table_path -Encoding ascii
+
+    Write-Host ""
+    cat $table_path
+
+    Write-Host "##vso[task.addattachment type=Distributedtask.Core.Summary;name=$result_table_name issue summary;]$table_path"
+}
+
+foreach ( $triplet in $complete_results.Keys) {
+    $combined_results = $complete_results[$triplet]
+    if ( $failurelogDir -ne "") {
+        save_failure_logs -combined_results $combined_results
+    }
+
+    write_xunit_results -combined_results $combined_results
+}
+
+
+# emit error last.  Unlike the table output this is going to be seen in the "status" section of the pipeline
+# and needs to be formatted for a single line.
+if ($errorOnRegression) {
+    write_errors_for_summary -complete_results $complete_results
+
+    if ($missing_triplets.Count -ne 0) {
+        $regression_log_directory = Join-Path "$outputDir" "failureLogs"
+        if ( -not (Test-Path $regression_log_directory)) {
+            mkdir $regression_log_directory | Out-Null
+        }
+        $file_path = Join-Path $regression_log_directory "missing_test_results.txt"
+        $message = "Test logs are missing for the following triplets: $($hash.Keys | %{"$($_)($($hash[$_]))"})`n"
+        $message += "Without this information the we are unable to determine if the build has regressions. `n"
+        $message += "Missing test logs are sometimes the result of failures in the pipeline infrastructure. `n"
+        $message += "If you beleave this is the case please alert a member of the vcpkg team to investigate. `n"
+        $message | Out-File $file_path -Encoding ascii
+    }
+}

scripts/azure-pipelines/azure-pipelines.yml

@@ -0,0 +1,56 @@
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: MIT
+#
+
+variables:
+  clean-tombstones: 'false'
+stages:
+  - stage: Clean_Tombstones
+    displayName: 'Clean Tombstones'
+    pool:
+      name: PrWin-2020-04-21-1
+    jobs:
+      - job:
+        displayName: 'Clean Tombstones'
+        condition: eq(variables['clean-tombstones'], 'true')
+        timeoutInMinutes: 10
+        steps:
+        - task: PowerShell@2
+          displayName: 'Initialize Environment'
+          inputs:
+            filePath: 'scripts/azure-pipelines/windows/initialize-environment.ps1'
+        - powershell: |
+            Remove-Item archives\fail -Force -Recurse
+          displayName: 'Delete archives\fail'
+
+  - stage: Build
+    jobs:
+      - template: windows/azure-pipelines.yml
+        parameters:
+          triplet: x86-windows
+          jobName: x86_windows
+
+      - template: windows/azure-pipelines.yml
+        parameters:
+          triplet: x64-windows
+          jobName: x64_windows
+
+      - template: windows/azure-pipelines.yml
+        parameters:
+          triplet: x64-windows-static
+          jobName: x64_windows_static
+
+      - template: windows/azure-pipelines.yml
+        parameters:
+          triplet: x64-uwp
+          jobName: x64_uwp
+
+      - template: windows/azure-pipelines.yml
+        parameters:
+          triplet: arm64-windows
+          jobName: arm64_windows
+
+      - template: windows/azure-pipelines.yml
+        parameters:
+          triplet: arm-uwp
+          jobName: arm_uwp

scripts/azure-pipelines/generate-skip-list.ps1

@@ -0,0 +1,72 @@
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: MIT
+#
+
+<#
+.SYNOPSIS
+Generates a list of ports to skip in the CI.
+
+.DESCRIPTION
+generate-skip-list takes a triplet, and the path to the ci.baseline.txt
+file, and generates a skip list string to pass to vcpkg.
+
+.PARAMETER Triplet
+The triplet to find skipped ports for.
+
+.PARAMETER BaselineFile
+The path to the ci.baseline.txt file.
+#>
+[CmdletBinding()]
+Param(
+    [string]$Triplet,
+    [string]$BaselineFile
+)
+
+$ErrorActionPreference = 'Stop'
+
+if (-not (Test-Path -Path $BaselineFile)) {
+    Write-Error "Unable to find baseline file $BaselineFile"
+}
+
+#read in the file, strip out comments and blank lines and spaces
+$baselineListRaw = Get-Content -Path $BaselineFile `
+    | Where-Object { -not ($_ -match "\s*#") } `
+    | Where-Object { -not ( $_ -match "^\s*$") } `
+    | ForEach-Object { $_ -replace "\s" }
+
+###############################################################
+# This script is running at the beginning of the CI test, so do a little extra
+# checking so things can fail early.
+
+#verify everything has a valid value
+$missingValues = $baselineListRaw | Where-Object { -not ($_ -match "=\w") }
+
+if ($missingValues) {
+    Write-Error "The following are missing values: $missingValues"
+}
+
+$invalidValues = $baselineListRaw `
+    | Where-Object { -not ($_ -match "=(skip|pass|fail|ignore)$") }
+
+if ($invalidValues) {
+    Write-Error "The following have invalid values: $invalidValues"
+}
+
+$baselineForTriplet = $baselineListRaw `
+    | Where-Object { $_ -match ":$Triplet=" }
+
+# Verify there are no duplicates (redefinitions are not allowed)
+$file_map = @{ }
+foreach ($port in $baselineForTriplet | ForEach-Object { $_ -replace ":.*$" }) {
+    if ($null -ne $file_map[$port]) {
+        Write-Error `
+            "$($port):$($Triplet) has multiple definitions in $baselineFile"
+    }
+    $file_map[$port] = $true
+}
+
+# Format the skip list for the command line
+$skip_list = $baselineForTriplet `
+    | Where-Object { $_ -match "=skip$" } `
+    | ForEach-Object { $_ -replace ":.*$" }
+[string]::Join(",", $skip_list)

scripts/azure-pipelines/windows/azure-pipelines.yml

@@ -0,0 +1,66 @@
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: MIT
+#
+
+jobs:
+- job: ${{ parameters.jobName }}
+  pool:
+    name: PrWin-2020-04-21-1
+
+  variables:
+    triplet: '${{ parameters.triplet }}'
+
+  timeoutInMinutes: 1440 # 1 day
+
+  steps:
+  - task: PowerShell@2
+    displayName: 'Initialize Environment'
+    inputs:
+      filePath: 'scripts/azure-pipelines/windows/initialize-environment.ps1'
+
+  - powershell: |
+      $baselineFile = "$(System.DefaultWorkingDirectory)\scripts\ci.baseline.txt"
+      $skipList = $(System.DefaultWorkingDirectory)\scripts\azure-pipelines\generate-skip-list.ps1 -Triplet "$(triplet)" -BaselineFile $baselineFile
+      Write-Host "baseline file: $baselineFile"
+      Write-Host "skip list: $skipList"
+      $(System.DefaultWorkingDirectory)\scripts\azure-pipelines\windows\ci-step.ps1 -Triplet "$(triplet)" -ExcludePorts $skipList
+      Write-Host "CI test script is complete"
+    errorActionPreference: continue
+    displayName: '** Build vcpkg and test ports **'
+
+  - powershell: |
+      $baseName = "$(triplet)"
+      $outputPathRoot = "$(System.ArtifactsDirectory)\raw xml results"
+      if(-not (Test-Path $outputPathRoot))
+      {
+          Write-Host "creating $outputPathRoot"
+          mkdir $outputPathRoot | Out-Null
+      }
+
+      $xmlPath = "$(System.DefaultWorkingDirectory)\test-full-ci.xml"
+      $outputXmlPath = "$outputPathRoot\$baseName.xml"
+
+      cp $xmlPath $(Build.ArtifactStagingDirectory)
+      Move-Item $xmlPath -Destination $outputXmlPath
+
+      # already in DevOps, no need for extra copies
+      rm $(System.DefaultWorkingDirectory)\console-out.txt -ErrorAction Ignore
+
+      Remove-Item "$(System.DefaultWorkingDirectory)\buildtrees\*" -Recurse -errorAction silentlycontinue
+      Remove-Item "$(System.DefaultWorkingDirectory)\packages\*" -Recurse -errorAction silentlycontinue
+      Remove-Item "$(System.DefaultWorkingDirectory)\installed\*" -Recurse -errorAction silentlycontinue
+    displayName: 'Collect logs and cleanup build'
+
+  - task: PowerShell@2
+    displayName: 'Analyze results and prepare test logs'
+    inputs:
+      failOnStderr: true
+      filePath: 'scripts/azure-pipelines/analyze-test-results.ps1'
+      arguments: '-baselineFile ''$(System.DefaultWorkingDirectory)\scripts\ci.baseline.txt'' -logDir ''$(System.ArtifactsDirectory)\raw xml results'' -failurelogDir ''archives\fail'' -outputDir ''$(Build.ArtifactStagingDirectory)'' -errorOnRegression -triplets ''$(triplet)'''
+
+  - task: PublishBuildArtifacts@1
+    displayName: 'Publish Artifact: $(triplet) port build failure logs'
+    inputs:
+      PathtoPublish: '$(Build.ArtifactStagingDirectory)\failureLogs'
+      ArtifactName: '$(triplet) port build failure logs'
+    condition: failed()

scripts/azure-pipelines/windows/ci-step.ps1

@@ -0,0 +1,163 @@
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: MIT
+#
+
+<#
+.SYNOPSIS
+Runs the bootstrap and port install parts of the vcpkg CI for Windows
+
+.DESCRIPTION
+There are multiple steps to the vcpkg CI; this is the most important one.
+First, it runs `boostrap-vcpkg.bat` in order to build the tool itself; it
+then installs either all of the ports specified, or all of the ports excluding
+those which are passed in $ExcludePorts. Then, it runs `vcpkg ci` to access the
+data, and prints all of the failures and successes to the Azure console.
+
+.PARAMETER Triplet
+The triplet to run the installs for -- one of the triplets known by vcpkg, like
+`x86-windows` and `x64-windows`.
+
+.PARAMETER OnlyIncludePorts
+The set of ports to install.
+
+.PARAMETER ExcludePorts
+If $OnlyIncludePorts is not passed, this set of ports is used to exclude ports to
+install from the set of all ports.
+
+.PARAMETER AdditionalVcpkgFlags
+Flags to pass to vcpkg in addition to the ports to install, and the triplet.
+#>
+[CmdletBinding()]
+param(
+    [Parameter(Mandatory = $true)][string]$Triplet,
+    [string]$OnlyIncludePorts = '',
+    [string]$ExcludePorts = '',
+    [string]$AdditionalVcpkgFlags = ''
+)
+
+Set-StrictMode -Version Latest
+
+$scriptsDir = Split-Path -parent $script:MyInvocation.MyCommand.Definition
+
+<#
+.SYNOPSIS
+Gets the first parent directory D of $startingDir such that D/$filename is a file.
+
+.DESCRIPTION
+Get-FileRecursivelyUp Looks for a directory containing $filename, starting in
+$startingDir, and then checking each parent directory of $startingDir in turn.
+It returns the first directory it finds.
+If the file is not found, the empty string is returned - this is likely to be
+a bug.
+
+.PARAMETER startingDir
+The directory to start looking for $filename in.
+
+.PARAMETER filename
+The filename to look for.
+#>
+function Get-FileRecursivelyUp() {
+    [CmdletBinding()]
+    param(
+        [Parameter(Mandatory = $true)][string]$startingDir,
+        [Parameter(Mandatory = $true)][string]$filename
+    )
+
+    $currentDir = $startingDir
+
+    while ($currentDir.Length -gt 0 -and -not (Test-Path "$currentDir\$filename")) {
+        Write-Verbose "Examining $currentDir for $filename"
+        $currentDir = Split-Path $currentDir -Parent
+    }
+
+    if ($currentDir.Length -eq 0) {
+        Write-Warning "None of $startingDir's parent directories contain $filename. This is likely a bug."
+    }
+
+    Write-Verbose "Examining $currentDir for $filename - Found"
+    return $currentDir
+}
+
+<#
+.SYNOPSIS
+Removes a file or directory, with backoff in the directory case.
+
+.DESCRIPTION
+Remove-Item -Recurse occasionally fails spuriously; in order to get around this,
+we remove with backoff. At a maximum, we will wait 180s before giving up.
+
+.PARAMETER Path
+The path to remove.
+#>
+function Remove-VcpkgItem {
+    [CmdletBinding()]
+    param([Parameter(Mandatory = $true)][string]$Path)
+
+    if ([string]::IsNullOrEmpty($Path)) {
+        return
+    }
+
+    if (Test-Path $Path) {
+        # Remove-Item -Recurse occasionally fails. This is a workaround
+        if ((Get-Item $Path) -is [System.IO.DirectoryInfo]) {
+            Remove-Item $Path -Force -Recurse -ErrorAction SilentlyContinue
+            for ($i = 0; $i -le 60 -and (Test-Path $Path); $i++) { # ~180s max wait time
+                Start-Sleep -m (100 * $i)
+                Remove-Item $Path -Force -Recurse -ErrorAction SilentlyContinue
+            }
+
+            if (Test-Path $Path) {
+                Write-Error "$Path was unable to be fully deleted."
+                throw;
+            }
+        }
+        else {
+            Remove-Item $Path -Force
+        }
+    }
+}
+
+$vcpkgRootDir = Get-FileRecursivelyUp $scriptsDir .vcpkg-root
+
+Write-Host "Bootstrapping vcpkg ..."
+& "$vcpkgRootDir\bootstrap-vcpkg.bat" -Verbose
+if (!$?) { throw "bootstrap failed" }
+Write-Host "Bootstrapping vcpkg ... done."
+
+$ciXmlPath = "$vcpkgRootDir\test-full-ci.xml"
+$consoleOuputPath = "$vcpkgRootDir\console-out.txt"
+Remove-VcpkgItem $ciXmlPath
+
+$env:VCPKG_FEATURE_FLAGS = "binarycaching"
+
+if (![string]::IsNullOrEmpty($OnlyIncludePorts)) {
+    ./vcpkg install --triplet $Triplet $OnlyIncludePorts $AdditionalVcpkgFlags `
+        "--x-xunit=$ciXmlPath" | Tee-Object -FilePath "$consoleOuputPath"
+}
+else {
+    $exclusions = ""
+    if (![string]::IsNullOrEmpty($ExcludePorts)) {
+        $exclusions = "--exclude=$ExcludePorts"
+    }
+
+    if ( $Triplet -notmatch "x86-windows" -and $Triplet -notmatch "x64-windows" ) {
+        # WORKAROUND: the x86-windows flavors of these are needed for all
+        # cross-compilation, but they are not auto-installed.
+        # Install them so the CI succeeds
+        ./vcpkg install "protobuf:x86-windows" "boost-build:x86-windows" "sqlite3:x86-windows"
+        if (-not $?) { throw "Failed to install protobuf & boost-build & sqlite3" }
+    }
+
+    # Turn all error messages into strings for output in the CI system.
+    # This is needed due to the way the public Azure DevOps turns error output to pipeline errors,
+    # even when told to ignore error output.
+    ./vcpkg ci $Triplet $AdditionalVcpkgFlags "--x-xunit=$ciXmlPath" $exclusions 2>&1 `
+    | ForEach-Object {
+        if ($_ -is [System.Management.Automation.ErrorRecord]) { $_.ToString() } else { $_ }
+    }
+
+    # Phasing out the console output (it is already saved in DevOps) Create a dummy file for now.
+    Set-Content -LiteralPath "$consoleOuputPath" -Value ''
+}
+
+Write-Host "CI test is complete"

scripts/azure-pipelines/windows/create-vmss.ps1

@@ -0,0 +1,458 @@
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: MIT
+#
+#
+
+<#
+.SYNOPSIS
+Creates a Windows virtual machine scale set, set up for vcpkg's CI.
+
+.DESCRIPTION
+create-vmss.ps1 creates an Azure Windows VM scale set, set up for vcpkg's CI
+system. See https://docs.microsoft.com/en-us/azure/virtual-machine-scale-sets/overview
+for more information.
+
+This script assumes you have installed Azure tools into PowerShell by following the instructions
+at https://docs.microsoft.com/en-us/powershell/azure/install-az-ps?view=azps-3.6.1
+or are running from Azure Cloud Shell.
+#>
+
+$Location = 'SouthCentralUS'
+$Prefix = 'PrWin-' + (Get-Date -Format 'yyyy-MM-dd')
+$VMSize = 'Standard_F16s_v2'
+$ProtoVMName = 'PROTOTYPE'
+$LiveVMPrefix = 'BUILD'
+$WindowsServerSku = '2019-Datacenter'
+$InstalledDiskSizeInGB = 1024
+$ErrorActionPreference = 'Stop'
+
+$ProgressActivity = 'Creating Scale Set'
+$TotalProgress = 12
+$CurrentProgress = 1
+
+<#
+.SYNOPSIS
+Returns whether there's a name collision in the resource group.
+
+.DESCRIPTION
+Find-ResourceGroupNameCollision takes a list of resources, and checks if $Test
+collides names with any of the resources.
+
+.PARAMETER Test
+The name to test.
+
+.PARAMETER Resources
+The list of resources.
+#>
+function Find-ResourceGroupNameCollision {
+  [CmdletBinding()]
+  Param([string]$Test, $Resources)
+
+  foreach ($resource in $Resources) {
+    if ($resource.ResourceGroupName -eq $Test) {
+      return $true
+    }
+  }
+
+  return $false
+}
+
+<#
+.SYNOPSIS
+Attempts to find a name that does not collide with any resources in the resource group.
+
+.DESCRIPTION
+Find-ResourceGroupName takes a set of resources from Get-AzResourceGroup, and finds the
+first name in {$Prefix, $Prefix-1, $Prefix-2, ...} such that the name doesn't collide with
+any of the resources in the resource group.
+
+.PARAMETER Prefix
+The prefix of the final name; the returned name will be of the form "$Prefix(-[1-9][0-9]*)?"
+#>
+function Find-ResourceGroupName {
+  [CmdletBinding()]
+  Param([string] $Prefix)
+
+  $resources = Get-AzResourceGroup
+  $result = $Prefix
+  $suffix = 0
+  while (Find-ResourceGroupNameCollision -Test $result -Resources $resources) {
+    $suffix++
+    $result = "$Prefix-$suffix"
+  }
+
+  return $result
+}
+
+<#
+.SYNOPSIS
+Creates a randomly generated password.
+
+.DESCRIPTION
+New-Password generates a password, randomly, of length $Length, containing
+only alphanumeric characters (both uppercase and lowercase).
+
+.PARAMETER Length
+The length of the returned password.
+#>
+function New-Password {
+  Param ([int] $Length = 32)
+
+  $Chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"
+  $result = ''
+  for ($idx = 0; $idx -lt $Length; $idx++) {
+    # NOTE: this should probably use RNGCryptoServiceProvider
+    $result += $Chars[(Get-Random -Minimum 0 -Maximum $Chars.Length)]
+  }
+
+  return $result
+}
+
+<#
+.SYNOPSIS
+Waits for the shutdown of the specified resource.
+
+.DESCRIPTION
+Wait-Shutdown takes a VM, and checks if there's a 'PowerState/stopped'
+code; if there is, it returns. If there isn't, it waits ten seconds and
+tries again.
+
+.PARAMETER ResourceGroupName
+The name of the resource group to look up the VM in.
+
+.PARAMETER Name
+The name of the virtual machine to wait on.
+#>
+function Wait-Shutdown {
+  [CmdletBinding()]
+  Param([string]$ResourceGroupName, [string]$Name)
+
+  Write-Host "Waiting for $Name to stop..."
+  while ($true) {
+    $Vm = Get-AzVM -ResourceGroupName $ResourceGroupName -Name $Name -Status
+    $highestStatus = $Vm.Statuses.Count
+    for ($idx = 0; $idx -lt $highestStatus; $idx++) {
+      if ($Vm.Statuses[$idx].Code -eq 'PowerState/stopped') {
+        return
+      }
+    }
+
+    Write-Host "... not stopped yet, sleeping for 10 seconds"
+    Start-Sleep -Seconds 10
+  }
+}
+
+<#
+.SYNOPSIS
+Sanitizes a name to be used in a storage account.
+
+.DESCRIPTION
+Sanitize-Name takes a string, and removes all of the '-'s and
+lowercases the string, since storage account names must have no
+'-'s and must be completely lowercase alphanumeric. It then makes
+certain that the length of the string is not greater than 24,
+since that is invalid.
+
+.PARAMETER RawName
+The name to sanitize.
+#>
+function Sanitize-Name {
+  [CmdletBinding()]
+  Param(
+    [string]$RawName
+  )
+
+  $result = $RawName.Replace('-', '').ToLowerInvariant()
+  if ($result.Length -gt 24) {
+    Write-Error 'Sanitized name for storage account $result was too long.'
+  }
+
+  return $result
+}
+
+####################################################################################################
+Write-Progress `
+  -Activity $ProgressActivity `
+  -Status 'Creating resource group' `
+  -PercentComplete (100 / $TotalProgress * $CurrentProgress++)
+
+$ResourceGroupName = Find-ResourceGroupName $Prefix
+$AdminPW = New-Password
+New-AzResourceGroup -Name $ResourceGroupName -Location $Location
+$AdminPWSecure = ConvertTo-SecureString $AdminPW -AsPlainText -Force
+$Credential = New-Object System.Management.Automation.PSCredential ("AdminUser", $AdminPWSecure)
+
+####################################################################################################
+Write-Progress `
+  -Activity $ProgressActivity `
+  -Status 'Creating virtual network' `
+  -PercentComplete (100 / $TotalProgress * $CurrentProgress++)
+
+$allowHttp = New-AzNetworkSecurityRuleConfig `
+  -Name AllowHTTP `
+  -Description 'Allow HTTP(S)' `
+  -Access Allow `
+  -Protocol Tcp `
+  -Direction Outbound `
+  -Priority 1008 `
+  -SourceAddressPrefix * `
+  -SourcePortRange * `
+  -DestinationAddressPrefix * `
+  -DestinationPortRange @(80, 443)
+
+$allowDns = New-AzNetworkSecurityRuleConfig `
+  -Name AllowDNS `
+  -Description 'Allow DNS' `
+  -Access Allow `
+  -Protocol * `
+  -Direction Outbound `
+  -Priority 1009 `
+  -SourceAddressPrefix * `
+  -SourcePortRange * `
+  -DestinationAddressPrefix * `
+  -DestinationPortRange 53
+
+$allowStorage = New-AzNetworkSecurityRuleConfig `
+  -Name AllowStorage `
+  -Description 'Allow Storage' `
+  -Access Allow `
+  -Protocol * `
+  -Direction Outbound `
+  -Priority 1010 `
+  -SourceAddressPrefix VirtualNetwork `
+  -SourcePortRange * `
+  -DestinationAddressPrefix Storage `
+  -DestinationPortRange *
+
+$denyEverythingElse = New-AzNetworkSecurityRuleConfig `
+  -Name DenyElse `
+  -Description 'Deny everything else' `
+  -Access Deny `
+  -Protocol * `
+  -Direction Outbound `
+  -Priority 1011 `
+  -SourceAddressPrefix * `
+  -SourcePortRange * `
+  -DestinationAddressPrefix * `
+  -DestinationPortRange *
+
+$NetworkSecurityGroupName = $ResourceGroupName + 'NetworkSecurity'
+$NetworkSecurityGroup = New-AzNetworkSecurityGroup `
+  -Name $NetworkSecurityGroupName `
+  -ResourceGroupName $ResourceGroupName `
+  -Location $Location `
+  -SecurityRules @($allowHttp, $allowDns, $allowStorage, $denyEverythingElse)
+
+$SubnetName = $ResourceGroupName + 'Subnet'
+$Subnet = New-AzVirtualNetworkSubnetConfig `
+  -Name $SubnetName `
+  -AddressPrefix "10.0.0.0/16" `
+  -NetworkSecurityGroup $NetworkSecurityGroup
+
+$VirtualNetworkName = $ResourceGroupName + 'Network'
+$VirtualNetwork = New-AzVirtualNetwork `
+  -Name $VirtualNetworkName `
+  -ResourceGroupName $ResourceGroupName `
+  -Location $Location `
+  -AddressPrefix "10.0.0.0/16" `
+  -Subnet $Subnet
+
+####################################################################################################
+Write-Progress `
+  -Activity $ProgressActivity `
+  -Status 'Creating archives storage account' `
+  -PercentComplete (100 / $TotalProgress * $CurrentProgress++)
+
+$StorageAccountName = Sanitize-Name $ResourceGroupName
+
+New-AzStorageAccount `
+  -ResourceGroupName $ResourceGroupName `
+  -Location $Location `
+  -Name $StorageAccountName `
+  -SkuName 'Standard_LRS' `
+  -Kind StorageV2
+
+$StorageAccountKeys = Get-AzStorageAccountKey `
+  -ResourceGroupName $ResourceGroupName `
+  -Name $StorageAccountName
+
+$StorageAccountKey = $StorageAccountKeys[0].Value
+
+$StorageContext = New-AzStorageContext `
+  -StorageAccountName $StorageAccountName `
+  -StorageAccountKey $StorageAccountKey
+
+$ArchivesFiles = New-AzStorageShare -Name 'archives' -Context $StorageContext
+Set-AzStorageShareQuota -ShareName 'archives' -Context $StorageContext -Quota 5120
+$LogFiles = New-AzStorageShare -Name 'logs' -Context $StorageContext
+Set-AzStorageShareQuota -ShareName 'logs' -Context $StorageContext -Quota 64
+
+####################################################################################################
+Write-Progress `
+  -Activity 'Creating prototype VM' `
+  -PercentComplete (100 / $TotalProgress * $CurrentProgress++)
+
+$NicName = $ResourceGroupName + 'NIC'
+$Nic = New-AzNetworkInterface `
+  -Name $NicName `
+  -ResourceGroupName $ResourceGroupName `
+  -Location $Location `
+  -Subnet $VirtualNetwork.Subnets[0]
+
+$VM = New-AzVMConfig -Name $ProtoVMName -VMSize $VMSize
+$VM = Set-AzVMOperatingSystem `
+  -VM $VM `
+  -Windows `
+  -ComputerName $ProtoVMName `
+  -Credential $Credential `
+  -ProvisionVMAgent `
+  -EnableAutoUpdate
+
+$VM = Add-AzVMNetworkInterface -VM $VM -Id $Nic.Id
+$VM = Set-AzVMSourceImage `
+  -VM $VM `
+  -PublisherName 'MicrosoftWindowsServer' `
+  -Offer 'WindowsServer' `
+  -Skus $WindowsServerSku `
+  -Version latest
+
+$InstallDiskName = $ProtoVMName + "InstallDisk"
+$VM = Add-AzVMDataDisk `
+  -Vm $VM `
+  -Name $InstallDiskName `
+  -Lun 0 `
+  -Caching ReadWrite `
+  -CreateOption Empty `
+  -DiskSizeInGB $InstalledDiskSizeInGB `
+  -StorageAccountType 'StandardSSD_LRS'
+
+$VM = Set-AzVMBootDiagnostic -VM $VM -Disable
+New-AzVm `
+  -ResourceGroupName $ResourceGroupName `
+  -Location $Location `
+  -VM $VM
+
+####################################################################################################
+Write-Progress `
+  -Activity $ProgressActivity `
+  -Status 'Running provisioning script provision-image.ps1 in VM' `
+  -PercentComplete (100 / $TotalProgress * $CurrentProgress++)
+
+Invoke-AzVMRunCommand `
+  -ResourceGroupName $ResourceGroupName `
+  -VMName $ProtoVMName `
+  -CommandId 'RunPowerShellScript' `
+  -ScriptPath "$PSScriptRoot\provision-image.ps1" `
+  -Parameter @{AdminUserPassword = $AdminPW; `
+    StorageAccountName=$StorageAccountName; `
+    StorageAccountKey=$StorageAccountKey;}
+
+####################################################################################################
+Write-Progress `
+  -Activity $ProgressActivity `
+  -Status 'Restarting VM' `
+  -PercentComplete (100 / $TotalProgress * $CurrentProgress++)
+
+Restart-AzVM -ResourceGroupName $ResourceGroupName -Name $ProtoVMName
+
+####################################################################################################
+Write-Progress `
+  -Activity $ProgressActivity `
+  -Status 'Running provisioning script sysprep.ps1 in VM' `
+  -PercentComplete (100 / $TotalProgress * $CurrentProgress++)
+
+Invoke-AzVMRunCommand `
+  -ResourceGroupName $ResourceGroupName `
+  -VMName $ProtoVMName `
+  -CommandId 'RunPowerShellScript' `
+  -ScriptPath "$PSScriptRoot\sysprep.ps1"
+
+####################################################################################################
+Write-Progress `
+  -Activity $ProgressActivity `
+  -Status 'Waiting for VM to shut down' `
+  -PercentComplete (100 / $TotalProgress * $CurrentProgress++)
+
+Wait-Shutdown -ResourceGroupName $ResourceGroupName -Name $ProtoVMName
+
+####################################################################################################
+Write-Progress `
+  -Activity $ProgressActivity `
+  -Status 'Converting VM to Image' `
+  -PercentComplete (100 / $TotalProgress * $CurrentProgress++)
+
+Stop-AzVM `
+  -ResourceGroupName $ResourceGroupName `
+  -Name $ProtoVMName `
+  -Force
+
+Set-AzVM `
+  -ResourceGroupName $ResourceGroupName `
+  -Name $ProtoVMName `
+  -Generalized
+
+$VM = Get-AzVM -ResourceGroupName $ResourceGroupName -Name $ProtoVMName
+$PrototypeOSDiskName = $VM.StorageProfile.OsDisk.Name
+$ImageConfig = New-AzImageConfig -Location $Location -SourceVirtualMachineId $VM.ID
+$Image = New-AzImage -Image $ImageConfig -ImageName $ProtoVMName -ResourceGroupName $ResourceGroupName
+
+####################################################################################################
+Write-Progress `
+  -Activity $ProgressActivity `
+  -Status 'Deleting unused VM and disk' `
+  -PercentComplete (100 / $TotalProgress * $CurrentProgress++)
+
+Remove-AzVM -Id $VM.ID -Force
+Remove-AzDisk -ResourceGroupName $ResourceGroupName -DiskName $PrototypeOSDiskName -Force
+Remove-AzDisk -ResourceGroupName $ResourceGroupName -DiskName $InstallDiskName -Force
+
+####################################################################################################
+Write-Progress `
+  -Activity $ProgressActivity `
+  -Status 'Creating scale set' `
+  -PercentComplete (100 / $TotalProgress * $CurrentProgress++)
+
+$VmssIpConfigName = $ResourceGroupName + 'VmssIpConfig'
+$VmssIpConfig = New-AzVmssIpConfig -SubnetId $Nic.IpConfigurations[0].Subnet.Id -Primary -Name $VmssIpConfigName
+$VmssName = $ResourceGroupName + 'Vmss'
+$Vmss = New-AzVmssConfig `
+  -Location $Location `
+  -SkuCapacity 6 `
+  -SkuName $VMSize `
+  -SkuTier 'Standard' `
+  -Overprovision $false `
+  -UpgradePolicyMode Manual
+
+$Vmss = Add-AzVmssNetworkInterfaceConfiguration `
+  -VirtualMachineScaleSet $Vmss `
+  -Primary $true `
+  -IpConfiguration $VmssIpConfig `
+  -NetworkSecurityGroupId $NetworkSecurityGroup.Id `
+  -Name $NicName
+
+$Vmss = Set-AzVmssOsProfile `
+  -VirtualMachineScaleSet $Vmss `
+  -ComputerNamePrefix $LiveVMPrefix `
+  -AdminUsername 'AdminUser' `
+  -AdminPassword $AdminPW `
+  -WindowsConfigurationProvisionVMAgent $true `
+  -WindowsConfigurationEnableAutomaticUpdate $true
+
+$Vmss = Set-AzVmssStorageProfile `
+  -VirtualMachineScaleSet $Vmss `
+  -OsDiskCreateOption 'FromImage' `
+  -OsDiskCaching ReadWrite `
+  -ImageReferenceId $Image.Id
+
+New-AzVmss `
+  -ResourceGroupName $ResourceGroupName `
+  -Name $VmssName `
+  -VirtualMachineScaleSet $Vmss
+
+####################################################################################################
+Write-Progress -Activity $ProgressActivity -Completed
+Write-Host "Location: $Location"
+Write-Host "Resource group name: $ResourceGroupName"
+Write-Host "User name: AdminUser"
+Write-Host "Using generated password: $AdminPW"
+Write-Host 'Finished!'

scripts/azure-pipelines/windows/initialize-environment.ps1

@@ -0,0 +1,93 @@
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: MIT
+#
+<#
+.SYNOPSIS
+Sets up the environment to run other vcpkg CI steps in an Azure Pipelines job.
+
+.DESCRIPTION
+This script maps network drives from infrastructure and cleans out anything that
+might have been leftover from a previous run.
+
+.PARAMETER ForceAllPortsToRebuildKey
+A subdirectory / key to use to force a build without any previous run caching,
+if necessary.
+#>
+
+[CmdletBinding()]
+Param(
+    [string]$ForceAllPortsToRebuildKey = ''
+)
+
+$StorageAccountName = $env:StorageAccountName
+$StorageAccountKey = $env:StorageAccountKey
+
+function Remove-DirectorySymlink {
+    Param([string]$Path)
+    if (Test-Path $Path) {
+        [System.IO.Directory]::Delete($Path)
+    }
+}
+
+Write-Host 'Setting up archives mount'
+if (-Not (Test-Path W:)) {
+    net use W: "\\$StorageAccountName.file.core.windows.net\archives" /u:"AZURE\$StorageAccountName" $StorageAccountKey
+}
+
+Write-Host 'Setting up logs mount'
+if (-Not (Test-Path L:)) {
+    net use L: "\\$StorageAccountName.file.core.windows.net\logs" /u:"AZURE\$StorageAccountName" $StorageAccountKey
+}
+
+Write-Host 'Creating downloads directory'
+mkdir D:\downloads -ErrorAction SilentlyContinue
+
+# Delete entries in the downloads folder, except:
+#   those in the 'tools' folder
+#   those last accessed in the last 30 days
+Get-ChildItem -Path D:\downloads -Exclude "tools" `
+    | Where-Object{ $_.LastAccessTime -lt (get-Date).AddDays(-30) } `
+    | ForEach-Object{Remove-Item -Path $_ -Recurse -Force}
+
+# Msys sometimes leaves a database lock file laying around, especially if there was a failed job
+# which causes unrelated failures in jobs that run later on the machine.
+# work around this by just removing the vcpkg installed msys2 if it exists
+if( Test-Path D:\downloads\tools\msys2 )
+{
+    Write-Host "removing previously installed msys2"
+    Remove-Item D:\downloads\tools\msys2 -Recurse -Force
+}
+
+Write-Host 'Setting up archives path...'
+if ([string]::IsNullOrWhiteSpace($ForceAllPortsToRebuildKey))
+{
+    $archivesPath = 'W:\'
+}
+else
+{
+    $archivesPath = "W:\force\$ForceAllPortsToRebuildKey"
+    if (-Not (Test-Path $fullPath)) {
+        Write-Host 'Creating $archivesPath'
+        mkdir $archivesPath
+    }
+}
+
+Write-Host "Linking archives => $archivesPath"
+Remove-DirectorySymlink archives
+cmd /c "mklink /D archives $archivesPath"
+
+Write-Host 'Linking installed => E:\installed'
+Remove-DirectorySymlink installed
+Remove-Item E:\installed -Recurse -Force -ErrorAction SilentlyContinue
+mkdir E:\installed
+cmd /c "mklink /D installed E:\installed"
+
+Write-Host 'Linking downloads => D:\downloads'
+Remove-DirectorySymlink downloads
+cmd /c "mklink /D downloads D:\downloads"
+
+Write-Host 'Cleaning buildtrees'
+Remove-Item buildtrees\* -Recurse -Force -errorAction silentlycontinue
+
+Write-Host 'Cleaning packages'
+Remove-Item packages\* -Recurse -Force -errorAction silentlycontinue

scripts/azure-pipelines/windows/provision-image.ps1

@@ -0,0 +1,447 @@
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: MIT
+
+<#
+.SYNOPSIS
+Sets up a machine to be an image for a scale set.
+
+.DESCRIPTION
+provision-image.ps1 runs on an existing, freshly provisioned virtual machine,
+and sets that virtual machine up as a vcpkg build machine. After this is done,
+(outside of this script), we take that machine and make it an image to be copied
+for setting up new VMs in the scale set.
+
+This script must either be run as admin, or one must pass AdminUserPassword;
+if the script is run with AdminUserPassword, it runs itself again as an
+administrator.
+
+.PARAMETER AdminUserPassword
+The administrator user's password; if this is $null, or not passed, then the
+script assumes it's running on an administrator account.
+
+.PARAMETER StorageAccountName
+The name of the storage account. Stored in the environment variable %StorageAccountName%.
+Used by the CI system to access the global storage.
+
+.PARAMETER StorageAccountKey
+The key of the storage account. Stored in the environment variable %StorageAccountKey%.
+Used by the CI system to access the global storage.
+#>
+param(
+  [string]$AdminUserPassword = $null,
+  [string]$StorageAccountName = $null,
+  [string]$StorageAccountKey = $null
+)
+
+$ErrorActionPreference = 'Stop'
+
+<#
+.SYNOPSIS
+Gets a random file path in the temp directory.
+
+.DESCRIPTION
+Get-TempFilePath takes an extension, and returns a path with a random
+filename component in the temporary directory with that extension.
+
+.PARAMETER Extension
+The extension to use for the path.
+#>
+Function Get-TempFilePath {
+  Param(
+    [String]$Extension
+  )
+
+  if ([String]::IsNullOrWhiteSpace($Extension)) {
+    throw 'Missing Extension'
+  }
+
+  $tempPath = [System.IO.Path]::GetTempPath()
+  $tempName = [System.IO.Path]::GetRandomFileName() + '.' + $Extension
+  return Join-Path $tempPath $tempName
+}
+
+if (-not [string]::IsNullOrEmpty($AdminUserPassword)) {
+  Write-Host "AdminUser password supplied; switching to AdminUser"
+  $PsExecPath = Get-TempFilePath -Extension 'exe'
+  Write-Host "Downloading psexec to $PsExecPath"
+  & curl.exe -L -o $PsExecPath -s -S https://live.sysinternals.com/PsExec64.exe
+  $PsExecArgs = @(
+    '-u',
+    'AdminUser',
+    '-p',
+    $AdminUserPassword,
+    '-accepteula',
+    '-h',
+    'C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe',
+    '-ExecutionPolicy',
+    'Unrestricted',
+    '-File',
+    $PSCommandPath
+  )
+
+  if (-Not ([string]::IsNullOrWhiteSpace($StorageAccountName))) {
+    $PsExecArgs += '-StorageAccountName'
+    $PsExecArgs += $StorageAccountName
+  }
+
+  if (-Not ([string]::IsNullOrWhiteSpace($StorageAccountKey))) {
+    $PsExecArgs += '-StorageAccountKey'
+    $PsExecArgs += $StorageAccountKey
+  }
+
+  Write-Host "Executing $PsExecPath " + @PsExecArgs
+
+  $proc = Start-Process -FilePath $PsExecPath -ArgumentList $PsExecArgs -Wait -PassThru
+  Write-Host 'Cleaning up...'
+  Remove-Item $PsExecPath
+  exit $proc.ExitCode
+}
+
+$VisualStudioBootstrapperUrl = 'https://aka.ms/vs/16/release/vs_enterprise.exe'
+$Workloads = @(
+  'Microsoft.VisualStudio.Workload.NativeDesktop',
+  'Microsoft.VisualStudio.Workload.Universal',
+  'Microsoft.VisualStudio.Component.VC.Tools.x86.x64',
+  'Microsoft.VisualStudio.Component.VC.Tools.ARM',
+  'Microsoft.VisualStudio.Component.VC.Tools.ARM64',
+  'Microsoft.VisualStudio.Component.VC.ATL',
+  'Microsoft.VisualStudio.Component.VC.ATLMFC',
+  'Microsoft.VisualStudio.Component.VC.v141.x86.x64.Spectre',
+  'Microsoft.VisualStudio.Component.Windows10SDK.18362',
+  'Microsoft.Net.Component.4.8.SDK',
+  'Microsoft.Component.NetFX.Native'
+)
+
+$MpiUrl = 'https://download.microsoft.com/download/A/E/0/AE002626-9D9D-448D-8197-1EA510E297CE/msmpisetup.exe'
+
+$CudaUrl = 'https://developer.download.nvidia.com/compute/cuda/10.1/Prod/local_installers/cuda_10.1.243_426.00_win10.exe'
+$CudaFeatures = 'nvcc_10.1 cuobjdump_10.1 nvprune_10.1 cupti_10.1 gpu_library_advisor_10.1 memcheck_10.1 ' + `
+  'nvdisasm_10.1 nvprof_10.1 visual_profiler_10.1 visual_studio_integration_10.1 cublas_10.1 cublas_dev_10.1 ' + `
+  'cudart_10.1 cufft_10.1 cufft_dev_10.1 curand_10.1 curand_dev_10.1 cusolver_10.1 cusolver_dev_10.1 cusparse_10.1 ' + `
+  'cusparse_dev_10.1 nvgraph_10.1 nvgraph_dev_10.1 npp_10.1 npp_dev_10.1 nvrtc_10.1 nvrtc_dev_10.1 nvml_dev_10.1 ' + `
+  'occupancy_calculator_10.1 fortran_examples_10.1'
+
+$BinSkimUrl = 'https://www.nuget.org/api/v2/package/Microsoft.CodeAnalysis.BinSkim/1.6.0'
+
+$ErrorActionPreference = 'Stop'
+$ProgressPreference = 'SilentlyContinue'
+
+<#
+.SYNOPSIS
+Writes a message to the screen depending on ExitCode.
+
+.DESCRIPTION
+Since msiexec can return either 0 or 3010 successfully, in both cases
+we write that installation succeeded, and which exit code it exited with.
+If msiexec returns anything else, we write an error.
+
+.PARAMETER ExitCode
+The exit code that msiexec returned.
+#>
+Function PrintMsiExitCodeMessage {
+  Param(
+    $ExitCode
+  )
+
+  # 3010 is probably ERROR_SUCCESS_REBOOT_REQUIRED
+  if ($ExitCode -eq 0 -or $ExitCode -eq 3010) {
+    Write-Host "Installation successful! Exited with $ExitCode."
+  }
+  else {
+    Write-Error "Installation failed! Exited with $ExitCode."
+  }
+}
+
+<#
+.SYNOPSIS
+Install Visual Studio.
+
+.DESCRIPTION
+InstallVisualStudio takes the $Workloads array, and installs it with the
+installer that's pointed at by $BootstrapperUrl.
+
+.PARAMETER Workloads
+The set of VS workloads to install.
+
+.PARAMETER BootstrapperUrl
+The URL of the Visual Studio installer, i.e. one of vs_*.exe.
+
+.PARAMETER InstallPath
+The path to install Visual Studio at.
+
+.PARAMETER Nickname
+The nickname to give the installation.
+#>
+Function InstallVisualStudio {
+  Param(
+    [String[]]$Workloads,
+    [String]$BootstrapperUrl,
+    [String]$InstallPath = $null,
+    [String]$Nickname = $null
+  )
+
+  try {
+    Write-Host 'Downloading Visual Studio...'
+    [string]$bootstrapperExe = Get-TempFilePath -Extension 'exe'
+    curl.exe -L -o $bootstrapperExe -s -S $BootstrapperUrl
+    Write-Host "Installing Visual Studio..."
+    $args = @('/c', $bootstrapperExe, '--quiet', '--norestart', '--wait', '--nocache')
+    foreach ($workload in $Workloads) {
+      $args += '--add'
+      $args += $workload
+    }
+
+    if (-not ([String]::IsNullOrWhiteSpace($InstallPath))) {
+      $args += '--installpath'
+      $args += $InstallPath
+    }
+
+    if (-not ([String]::IsNullOrWhiteSpace($Nickname))) {
+      $args += '--nickname'
+      $args += $Nickname
+    }
+
+    $proc = Start-Process -FilePath cmd.exe -ArgumentList $args -Wait -PassThru
+    PrintMsiExitCodeMessage $proc.ExitCode
+  }
+  catch {
+    Write-Error "Failed to install Visual Studio! $($_.Exception.Message)"
+  }
+}
+
+<#
+.SYNOPSIS
+Install a .msi file.
+
+.DESCRIPTION
+InstallMSI takes a url where an .msi lives, and installs that .msi to the system.
+
+.PARAMETER Name
+The name of the thing to install.
+
+.PARAMETER Url
+The URL at which the .msi lives.
+#>
+Function InstallMSI {
+  Param(
+    [String]$Name,
+    [String]$Url
+  )
+
+  try {
+    Write-Host "Downloading $Name..."
+    [string]$msiPath = Get-TempFilePath -Extension 'msi'
+    curl.exe -L -o $msiPath -s -S $Url
+    Write-Host "Installing $Name..."
+    $args = @('/i', $msiPath, '/norestart', '/quiet', '/qn')
+    $proc = Start-Process -FilePath 'msiexec.exe' -ArgumentList $args -Wait -PassThru
+    PrintMsiExitCodeMessage $proc.ExitCode
+  }
+  catch {
+    Write-Error "Failed to install $Name! $($_.Exception.Message)"
+  }
+}
+
+<#
+.SYNOPSIS
+Unpacks a zip file to $Dir.
+
+.DESCRIPTION
+InstallZip takes a URL of a zip file, and unpacks the zip file to the directory
+$Dir.
+
+.PARAMETER Name
+The name of the tool being installed.
+
+.PARAMETER Url
+The URL of the zip file to unpack.
+
+.PARAMETER Dir
+The directory to unpack the zip file to.
+#>
+Function InstallZip {
+  Param(
+    [String]$Name,
+    [String]$Url,
+    [String]$Dir
+  )
+
+  try {
+    Write-Host "Downloading $Name..."
+    [string]$zipPath = Get-TempFilePath -Extension 'zip'
+    curl.exe -L -o $zipPath -s -S $Url
+    Write-Host "Installing $Name..."
+    Expand-Archive -Path $zipPath -DestinationPath $Dir -Force
+  }
+  catch {
+    Write-Error "Failed to install $Name! $($_.Exception.Message)"
+  }
+}
+
+<#
+.SYNOPSIS
+Installs MPI
+
+.DESCRIPTION
+Downloads the MPI installer located at $Url, and installs it with the
+correct flags.
+
+.PARAMETER Url
+The URL of the installer.
+#>
+Function InstallMpi {
+  Param(
+    [String]$Url
+  )
+
+  try {
+    Write-Host 'Downloading MPI...'
+    [string]$installerPath = Get-TempFilePath -Extension 'exe'
+    curl.exe -L -o $installerPath -s -S $Url
+    Write-Host 'Installing MPI...'
+    $proc = Start-Process -FilePath $installerPath -ArgumentList @('-force', '-unattend') -Wait -PassThru
+    $exitCode = $proc.ExitCode
+    if ($exitCode -eq 0) {
+      Write-Host 'Installation successful!'
+    }
+    else {
+      Write-Error "Installation failed! Exited with $exitCode."
+    }
+  }
+  catch {
+    Write-Error "Failed to install MPI! $($_.Exception.Message)"
+  }
+}
+
+<#
+.SYNOPSIS
+Installs NVIDIA's CUDA Toolkit.
+
+.DESCRIPTION
+InstallCuda installs the CUDA Toolkit with the features specified as a
+space separated list of strings in $Features.
+
+.PARAMETER Url
+The URL of the CUDA installer.
+
+.PARAMETER Features
+A space-separated list of features to install.
+#>
+Function InstallCuda {
+  Param(
+    [String]$Url,
+    [String]$Features
+  )
+
+  try {
+    Write-Host 'Downloading CUDA...'
+    [string]$installerPath = Get-TempFilePath -Extension 'exe'
+    curl.exe -L -o $installerPath -s -S $Url
+    Write-Host 'Installing CUDA...'
+    $proc = Start-Process -FilePath $installerPath -ArgumentList @('-s ' + $Features) -Wait -PassThru
+    $exitCode = $proc.ExitCode
+    if ($exitCode -eq 0) {
+      Write-Host 'Installation successful!'
+    }
+    else {
+      Write-Error "Installation failed! Exited with $exitCode."
+    }
+  }
+  catch {
+    Write-Error "Failed to install CUDA! $($_.Exception.Message)"
+  }
+}
+
+<#
+.SYNOPSIS
+Partitions a new physical disk.
+
+.DESCRIPTION
+Takes the disk $DiskNumber, turns it on, then partitions it for use with label
+$Label and drive letter $Letter.
+
+.PARAMETER DiskNumber
+The number of the disk to set up.
+
+.PARAMETER Letter
+The drive letter at which to mount the disk.
+
+.PARAMETER Label
+The label to give the disk.
+#>
+Function New-PhysicalDisk {
+  Param(
+    [int]$DiskNumber,
+    [string]$Letter,
+    [string]$Label
+  )
+
+  if ($Letter.Length -ne 1) {
+    throw "Bad drive letter $Letter, expected only one letter. (Did you accidentially add a : ?)"
+  }
+
+  try {
+    Write-Host "Attempting to online physical disk $DiskNumber"
+    [string]$diskpartScriptPath = Get-TempFilePath -Extension 'txt'
+    [string]$diskpartScriptContent =
+    "SELECT DISK $DiskNumber`r`n" +
+    "ONLINE DISK`r`n"
+
+    Write-Host "Writing diskpart script to $diskpartScriptPath with content:"
+    Write-Host $diskpartScriptContent
+    Set-Content -Path $diskpartScriptPath -Value $diskpartScriptContent
+    Write-Host 'Invoking DISKPART...'
+    & diskpart.exe /s $diskpartScriptPath
+
+    Write-Host "Provisioning physical disk $DiskNumber as drive $Letter"
+    [string]$diskpartScriptContent =
+    "SELECT DISK $DiskNumber`r`n" +
+    "ATTRIBUTES DISK CLEAR READONLY`r`n" +
+    "CREATE PARTITION PRIMARY`r`n" +
+    "FORMAT FS=NTFS LABEL=`"$Label`" QUICK`r`n" +
+    "ASSIGN LETTER=$Letter`r`n"
+    Write-Host "Writing diskpart script to $diskpartScriptPath with content:"
+    Write-Host $diskpartScriptContent
+    Set-Content -Path $diskpartScriptPath -Value $diskpartScriptContent
+    Write-Host 'Invoking DISKPART...'
+    & diskpart.exe /s $diskpartScriptPath
+  }
+  catch {
+    Write-Error "Failed to provision physical disk $DiskNumber as drive $Letter! $($_.Exception.Message)"
+  }
+}
+
+Write-Host "AdminUser password not supplied; assuming already running as AdminUser"
+
+New-PhysicalDisk -DiskNumber 2 -Letter 'E' -Label 'install disk'
+
+Write-Host 'Disabling pagefile...'
+wmic computersystem set AutomaticManagedPagefile=False
+wmic pagefileset delete
+
+Write-Host 'Configuring AntiVirus exclusions...'
+Add-MPPreference -ExclusionPath C:\
+Add-MPPreference -ExclusionPath D:\
+Add-MPPreference -ExclusionPath E:\
+Add-MPPreference -ExclusionProcess ninja.exe
+Add-MPPreference -ExclusionProcess clang-cl.exe
+Add-MPPreference -ExclusionProcess cl.exe
+Add-MPPreference -ExclusionProcess link.exe
+Add-MPPreference -ExclusionProcess python.exe
+
+InstallVisualStudio -Workloads $Workloads -BootstrapperUrl $VisualStudioBootstrapperUrl -Nickname 'Stable'
+InstallMpi -Url $MpiUrl
+InstallCuda -Url $CudaUrl -Features $CudaFeatures
+InstallZip -Url $BinSkimUrl -Name 'BinSkim' -Dir 'C:\BinSkim'
+if (-Not ([string]::IsNullOrWhiteSpace($StorageAccountName))) {
+  Write-Host 'Storing storage account name to environment'
+  Set-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Control\Session Manager\Environment' `
+    -Name StorageAccountName `
+    -Value $StorageAccountName
+}
+if (-Not ([string]::IsNullOrWhiteSpace($StorageAccountKey))) {
+  Write-Host 'Storing storage account key to environment'
+  Set-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Control\Session Manager\Environment' `
+    -Name StorageAccountKey `
+    -Value $StorageAccountKey
+}

scripts/azure-pipelines/windows/sysprep.ps1

@@ -0,0 +1,17 @@
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: MIT
+#
+
+<#
+.SYNOPSIS
+Prepares the virtual machine for imaging.
+
+.DESCRIPTION
+Runs the `sysprep` utility to prepare the system for imaging.
+See https://docs.microsoft.com/en-us/windows-hardware/manufacture/desktop/sysprep--system-preparation--overview
+for more information.
+#>
+
+$ErrorActionPreference = 'Stop'
+Write-Host 'Running sysprep'
+& C:\Windows\system32\sysprep\sysprep.exe /oobe /generalize /shutdown

scripts/ci.baseline.txt

@@ -45,10 +45,14 @@
 7zip:x64-osx=fail
 7zip:x64-uwp=fail
 abseil:arm-uwp=fail
-ace:arm64-windows=fail
+# ace is failing because the port's attempt to make yasm available is not succeeding
 ace:arm-uwp=fail
+ace:arm64-windows=fail
 ace:x64-osx=fail
 ace:x64-uwp=fail
+ace:x64-windows-static=fail
+ace:x64-windows=fail
+ace:x86-windows=fail
 activemq-cpp:x64-windows-static=fail
 akali:x64-uwp=fail
 akali:arm-uwp=fail
@@ -270,7 +274,7 @@ cppcms:x64-osx=fail
 cppcms:x64-windows-static=fail
 cppfs:arm-uwp=fail
 cppfs:x64-uwp=fail
-cppgraphqlgen:arm-uwp=fail
+cppgraphqlgen:arm-uwp=ignore
 cppgraphqlgen:x64-uwp=ignore
 cppkafka:x64-linux=ignore
 cppmicroservices:arm64-windows=fail
@@ -305,8 +309,8 @@ cudnn:x64-windows-static=fail
 cudnn:x86-windows=fail
 date:arm64-windows=fail
 dbow2:x64-osx=fail
-dcmtk:arm64-windows=fail
 dcmtk:arm-uwp=fail
+dcmtk:arm64-windows=fail
 dcmtk:x64-uwp=fail
 detours:x64-linux=fail
 detours:x64-osx=fail
@@ -337,6 +341,7 @@ dlfcn-win32:x64-linux=fail
 dlfcn-win32:x64-osx=fail
 dlfcn-win32:x64-uwp=fail
 dmlc:arm-uwp=fail
+dmlc:arm64-windows=ignore
 dmlc:x64-uwp=fail
 dmlc:x64-windows-static=ignore
 dmlc:x86-windows=ignore
@@ -435,11 +440,11 @@ fdlibm:arm-uwp=fail
 fdlibm:x64-uwp=fail
 fftw3:arm-uwp=fail
 fftw3:x64-uwp=fail
+# ffmpeg on arm64 is currently failing due to an internal compiler error
+ffmpeg:arm64-windows=fail
 field3d:x64-windows=fail
 field3d:x64-windows-static=fail
 field3d:x86-windows=fail
-fizz:x64-windows=fail
-fizz:x64-windows-static=fail
 flint:x64-linux=fail
 flint:x64-osx=fail
 fltk:arm-uwp=fail
@@ -457,11 +462,6 @@ fmilib:x64-uwp=fail
 fmilib:x64-windows=ignore
 fmilib:x64-windows-static=ignore
 fmilib:x86-windows=ignore
-# Folly fails due to a compiler bug in MSVC 19.22.27905, fixed in newer releases
-folly:arm64-windows=fail
-folly:x86-windows=fail
-folly:x64-windows=fail
-folly:x64-windows-static=fail
 foonathan-memory:arm64-windows=fail
 foonathan-memory:arm-uwp=fail
 foonathan-memory:x64-uwp=fail
@@ -602,6 +602,7 @@ hwloc:x64-linux=fail
 hwloc:x64-osx=fail
 hwloc:x64-uwp=fail
 hyperscan:x64-linux=ignore
+# hypre has a conflict with 'superlu' port
 hypre:x64-linux=fail
 hypre:x64-osx=fail
 icu:arm64-windows=fail
@@ -957,10 +958,15 @@ libuuid:x86-windows=fail
 libuv:arm64-windows=fail
 libuv:arm-uwp=fail
 libuv:x64-uwp=fail
-libvpx:arm64-windows=fail
+# libvpx is failing because the port's attempt to make yasm available is not succeeding
 libvpx:arm-uwp=fail
+libvpx:arm64-windows=fail
 libvpx:x64-linux=fail
 libvpx:x64-osx=fail
+libvpx:x64-uwp=fail
+libvpx:x64-windows-static=fail
+libvpx:x64-windows=fail
+libvpx:x86-windows=fail
 libwandio:x86-windows=fail
 libwandio:x64-windows=fail
 libwandio:x64-windows-static=fail
@@ -995,9 +1001,6 @@ llvm:arm64-windows=fail
 llvm:arm-uwp=fail
 llvm:x64-uwp=fail
 llvm:x64-linux=ignore
-# disable them temporarily and wait for fix
-llvm:x64-windows=fail
-llvm:x64-windows-static=fail
 # installing iconv makes building llvm fail; needs to be fixed
 llvm:x64-osx=ignore
 lmdb:arm64-windows=fail
@@ -1122,8 +1125,12 @@ mozjpeg:x64-uwp            = skip
 mozjpeg:x64-windows        = skip
 mozjpeg:x64-windows-static = skip
 mozjpeg:x86-windows        = skip
+# mpg123 is failing because the port's attempt to make yasm available is not succeeding
 mpg123:arm-uwp=fail
 mpg123:x64-uwp=fail
+mpg123:x64-windows-static=fail
+mpg123:x64-windows=fail
+mpg123:x86-windows=fail
 mpir:arm64-windows=fail
 mpir:arm-uwp=fail
 mpir:x64-uwp=fail
@@ -1413,9 +1420,13 @@ polyhook2:x64-linux=fail
 polyhook2:x64-uwp=fail
 polyhook2:x64-osx=fail
 portable-snippets:arm-uwp=fail
-portaudio:arm64-windows=fail
+# Portaudio was broken by Ninja 1.9.0 https://github.com/ninja-build/ninja/pull/1406
 portaudio:arm-uwp=fail
+portaudio:arm64-windows=fail
 portaudio:x64-uwp=fail
+portaudio:x64-windows-static=fail
+portaudio:x64-windows=fail
+portaudio:x86-windows=fail
 portmidi:arm64-windows=fail
 portmidi:arm-uwp=fail
 portmidi:x64-linux=fail
@@ -1435,6 +1446,9 @@ protobuf-c:x64-windows-static=fail
 protobuf-c:x64-uwp=fail
 protobuf-c:arm64-windows=fail
 protobuf-c:arm-uwp=fail
+# proxygen fails with "Target 'Windows' not supported by proxygen!"
+proxygen:x64-windows=fail
+proxygen:x64-windows-static=fail
 ptex:arm-uwp=fail
 ptex:x64-linux=fail
 ptex:x64-osx=fail
@@ -1459,8 +1473,40 @@ qhull:x64-uwp=ignore
 qpid-proton:arm-uwp=fail
 qpid-proton:x64-uwp=fail
 qpid-proton:x64-windows-static=fail
+#qt5-activeqt is skipped because it conflicts with qt5-declarative:
+# Starting package 142/820: qt5-declarative:x86-windows
+# Building package qt5-declarative[core]:x86-windows...
+# Using cached binary package: C:\agent\_work\1\s\archives\94\9428e63fede20a51ac0631a9d94d8773e593dd06.zip
+# Building package qt5-declarative[core]:x86-windows... done
+# Installing package qt5-declarative[core]:x86-windows...
+#     The following files are already installed in C:/agent/_work/1/s/installed/x86-windows and are in conflict with qt5-declarative:x86-windows
+#
+# Installed by qt5-activeqt:x86-windows
+#     tools/qt5/bin/Qt5Gui.dll
+#     tools/qt5/bin/Qt5Widgets.dll
+#     tools/qt5/bin/bz2.dll
+#     tools/qt5/bin/freetype.dll
+#     tools/qt5/bin/glib-2.dll
+#     tools/qt5/bin/harfbuzz.dll
+#     tools/qt5/bin/jpeg62.dll
+#     tools/qt5/bin/libcharset.dll
+#     tools/qt5/bin/libiconv.dll
+#     tools/qt5/bin/libintl.dll
+#     tools/qt5/bin/libpng16.dll
+#     tools/qt5/bin/pcre.dll
+#     tools/qt5/bin/plugins/imageformats/qgif.dll
+#     tools/qt5/bin/plugins/imageformats/qico.dll
+#     tools/qt5/bin/plugins/imageformats/qjpeg.dll
+#     tools/qt5/bin/plugins/platforms/qwindows.dll
+#     tools/qt5/bin/plugins/styles/qwindowsvistastyle.dll
+qt5-activeqt:arm-uwp=skip
+qt5-activeqt:arm64-windows=skip
 qt5-activeqt:x64-linux=fail
 qt5-activeqt:x64-osx=fail
+qt5-activeqt:x64-uwp=skip
+qt5-activeqt:x64-windows-static=skip
+qt5-activeqt:x64-windows=skip
+qt5-activeqt:x86-windows=skip
 qt5-macextras:x64-linux=fail
 qt5-macextras:x64-windows=fail
 qt5-macextras:x64-windows-static=fail
@@ -1524,7 +1570,6 @@ redis-plus-plus:x64-windows-static=fail
 redis-plus-plus:arm64-windows=fail
 replxx:arm-uwp=fail
 replxx:x64-uwp=fail
-replxx:arm64-windows=fail
 reproc:arm-uwp=fail
 reproc:x64-uwp=fail
 restbed:arm-uwp=fail
@@ -1561,7 +1606,6 @@ sciter:x64-uwp=fail
 sciter:x64-windows-static=fail
 scnlib:arm-uwp=fail
 scnlib:x64-uwp=fail
-scnlib:x86-windows=fail
 scylla-wrapper:arm64-windows=fail
 scylla-wrapper:arm-uwp=fail
 scylla-wrapper:x64-linux=fail
@@ -1584,6 +1628,8 @@ sdl2-mixer:x64-uwp=fail
 sdl2-net:arm-uwp=fail
 sdl2-net:x64-uwp=fail
 seal:arm-uwp=fail
+# https://github.com/microsoft/vcpkg/issues/10918
+seal:x64-windows-static=fail
 seal:x64-uwp=fail
 secp256k1:x64-linux=fail
 secp256k1:x64-osx=fail
@@ -1676,9 +1722,30 @@ stormlib:arm-uwp=fail
 stormlib:x64-uwp=fail
 stxxl:arm-uwp=fail
 stxxl:x64-uwp=fail
-superlu:arm64-windows=fail
-superlu:arm-uwp=fail
-superlu:x64-uwp=fail
+# Sundials was broken by Ninja 1.9.0 https://github.com/ninja-build/ninja/pull/1406
+sundials:arm64-windows=fail
+sundials:x64-windows=fail
+sundials:x86-windows=fail
+# Conflicts between ports:
+#The following files are already installed in C:/agent/_work/1/s/installed/x64-windows-static
+# and are in conflict with superlu:x64-windows-static
+#
+#Installed by hypre:x64-windows-static
+#    include/slu_Cnames.h
+#    include/slu_cdefs.h
+#    include/slu_dcomplex.h
+#    include/slu_ddefs.h
+#    include/slu_scomplex.h
+#    include/slu_sdefs.h
+#    include/slu_util.h
+#    include/slu_zdefs.h
+#    include/supermatrix.h
+superlu:arm-uwp=skip
+superlu:arm-windows=skip
+superlu:arm64-windows=skip
+superlu:x64-uwp=skip
+superlu:x64-windows-static=skip
+superlu:x64-windows=skip
 systemc:arm64-windows=fail
 systemc:arm-uwp=fail
 systemc:x64-uwp=fail
@@ -1907,4 +1974,3 @@ zkpp:arm-uwp=fail
 c4core:arm-uwp=fail
 c4core:arm64-windows=fail
 c4core:x64-osx=fail
-