mirror of
https://github.com/microsoft/vcpkg.git
synced 2024-12-27 02:11:58 +08:00
Update vcpkg-tool to 2022-03-25 (#23757)
* Update vcpkg-tool to 2022-03-24 * Hook up https://github.com/microsoft/vcpkg-tool/pull/345 * Hook up https://github.com/microsoft/vcpkg-tool/pull/442 * Update vcpkg-tool to 2022-03-25 * Analysis of failures. * [Most recent nightly build failed](https://dev.azure.com/vcpkg/public/_build/results?buildId=69427) * [Validation of this tool update failed](https://dev.azure.com/vcpkg/public/_build/results?buildId=69417) ## Common to both: PASSING, REMOVE FROM FAIL LIST: chartdir:x64-windows (.\scripts\ci.baseline.txt) PASSING, REMOVE FROM FAIL LIST: chartdir:x64-windows-static-md (.\scripts\ci.baseline.txt) PASSING, REMOVE FROM FAIL LIST: chartdir:x86-windows (.\scripts\ci.baseline.txt) Probably fixed by https://github.com/microsoft/vcpkg/pull/23701 PASSING, REMOVE FROM FAIL LIST: gmp:x64-uwp (.\scripts\ci.baseline.txt) PASSING, REMOVE FROM FAIL LIST: gmp:x64-windows-static-md (.\scripts\ci.baseline.txt) Probably fixed by https://github.com/microsoft/vcpkg/pull/23466 ? REGRESSION: colmap:x64-windows-static-md failed with BUILD_FAILED. If expected, add colmap:x64-windows-static-md=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt. I don't know exactly what changed. I observe that * this thing depends on a *lot* of stuff * on March 14 we didn't even attempt to build this * the x64-windows ones are already in the baseline so I skipped it. REGRESSION: qtdeclarative:x64-windows. If expected, add qtdeclarative:x64-windows=fail to .\scripts\ci.baseline.txt. REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt. REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt. REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt. REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt. REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt. REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt. REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt. This is a reporting change: The new world order also includes host build failures which is why it's duplicated. See also https://github.com/microsoft/vcpkg/issues/23714 See also https://github.com/microsoft/vcpkg/issues/23490 I'm nervous about baslining this because it seems most of the qt world is built on top of this port I filed https://github.com/microsoft/vcpkg/issues/23824 about this and @Neumann-A indicated this should be fixed by https://github.com/microsoft/vcpkg/pull/23755 REGRESSION: nettle:x64-uwp. If expected, add nettle:x64-uwp=fail to .\scripts\ci.baseline.txt. REGRESSION: nettle:x64-windows-static-md. If expected, add nettle:x64-windows-static-md=fail to .\scripts\ci.baseline.txt. REGRESSION: nettle:x64-uwp failed with BUILD_FAILED. If expected, add nettle:x64-uwp=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt. REGRESSION: nettle:x64-windows-static-md failed with POST_BUILD_CHECKS_FAILED. If expected, add nettle:x64-windows-static-md=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt. Didn't analyze, probably fixed by https://github.com/microsoft/vcpkg/pull/23519 ? REGRESSION: libgpg-error:x64-uwp. If expected, add libgpg-error:x64-uwp=fail to .\scripts\ci.baseline.txt. REGRESSION: libgpg-error:x64-uwp failed with BUILD_FAILED. If expected, add libgpg-error:x64-uwp=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt. This was broken by VS2022 update: ``` C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Microsoft\VisualStudio\v17.0\AppxPackage\Microsoft.AppXPackage.Targets(892,25): error MSB4086: A numeric comparison was attempted on "$(TargetPlatformMinVersion)" that evaluates to "" instead of a number, in condition "'$(TargetPlatformMinVersion)' >= '10.0.17200.0'". [C:\Dev\vcpkg\buildtrees\libgpg-error\x64-uwp-rel\error-1.42-2324ddbc71.clean\SMP\libgpg-error_winrt.vcxproj] ``` REGRESSION: libmikmod:x64-osx. If expected, add libmikmod:x64-osx=fail to .\scripts\ci.baseline.txt. REGRESSION: libmikmod:x64-osx failed with BUILD_FAILED. If expected, add libmikmod:x64-osx=fail to /Users/vagrant/Data/work/2/s/scripts/azure-pipelines/../ci.baseline.txt. Broken between [2022-03-16](https://dev.azure.com/vcpkg/public/_build/results?buildId=68947) and [2022-03-18](https://dev.azure.com/vcpkg/public/_build/results?buildId=69051). Unfortunately I don't see obvious reasons why. Nothing else depends on this and nobody has noticed in 2 weeks, so I'm baslining it for now. (Will investigate shortly...) ## Only broken in tool update: REGRESSION: mesa:x64-windows failed with BUILD_FAILED. If expected, add mesa:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt. ``` -- Downloading https://gitlab.freedesktop.org/mesa/mesa/-/archive/mesa-21.2.5/mesa-mesa-21.2.5.tar.gz -> mesa-mesa-mesa-21.2.5-1.tar.gz... -- Extracting source /Users/vagrant/Data/downloads/mesa-mesa-mesa-21.2.5-1.tar.gz -- Applying patch swravx512-post-static-link.patch -- Applying patch swr-msvc-2.patch -- Applying patch swr-llvm13.patch -- Applying patch radv-msvc-llvm13-2.patch -- Applying patch d3d10sw.patch -- Using source at /Users/vagrant/Data/buildtrees/mesa/src/esa-21.2.5-2df234d2b1.clean Traceback (most recent call last): File "<string>", line 1, in <module> ModuleNotFoundError: No module named 'mako' CMake Error at ports/mesa/portfile.cmake:85 (message): Python package 'mako' needs to be installed for port 'mesa'. Complete list of required python packages: setuptools;mako Call Stack (most recent call first): ports/mesa/portfile.cmake:91 (vcpkg_get_python_package) scripts/ports.cmake:145 (include) ``` Looks like this is being tracked by https://github.com/microsoft/vcpkg/pull/23089 ; perhaps that we don't have as aggressive a recycling strategy for macos boxes as we do for the others has let different machines give different results? ## Only broken without tool update: REGRESSION: chromium-base:x64-osx. If expected, add chromium-base:x64-osx=fail to .\scripts\ci.baseline.txt. This one has been constantly flaky; I baselined it. REGRESSION: libxml2:x64-osx. If expected, add libxml2:x64-osx=fail to .\scripts\ci.baseline.txt. This port uses vcpkg_from_git and the upstream server was down during the build. * Restore chartdir to the baseline, I thought https://github.com/microsoft/vcpkg/pull/23732 had been merged.
This commit is contained in:
Billy O'Neal
GitHub
parent
7e755c2ace
commit
671db8ae9d
@ -1,453 +0,0 @@
|
||||
# Copyright (c) Microsoft Corporation.
|
||||
# SPDX-License-Identifier: MIT
|
||||
#
|
||||
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Analyze the test results as output by the CI system.
|
||||
|
||||
.DESCRIPTION
|
||||
Takes the set of port test results from $logDir,
|
||||
and the baseline from $baselineFile, and makes certain that the set
|
||||
of failures we expected are exactly the set of failures we got.
|
||||
Then, uploads the logs from any unexpected failures.
|
||||
|
||||
.PARAMETER logDir
|
||||
Directory of xml test logs to analyze.
|
||||
|
||||
.PARAMETER allResults
|
||||
Include tests that have no change from the baseline in the output.
|
||||
|
||||
.PARAMETER triplet
|
||||
The triplet to analyze.
|
||||
|
||||
.PARAMETER baselineFile
|
||||
The path to the ci.baseline.txt file in the vcpkg repository.
|
||||
|
||||
.PARAMETER passingIsPassing
|
||||
Indicates that 'Passing, remove from fail list' results should not be emitted as failures. (For example, this is used
|
||||
when using vcpkg to test a prerelease MSVC++ compiler)
|
||||
#>
|
||||
[CmdletBinding()]
|
||||
Param(
|
||||
[Parameter(Mandatory = $true)]
|
||||
[string]$logDir,
|
||||
[switch]$allResults,
|
||||
[Parameter(Mandatory = $true)]
|
||||
[string]$triplet,
|
||||
[Parameter(Mandatory = $true)]
|
||||
[string]$baselineFile,
|
||||
[switch]$passingIsPassing = $false
|
||||
)
|
||||
|
||||
$ErrorActionPreference = 'Stop'
|
||||
|
||||
if ( -not (Test-Path $logDir) ) {
|
||||
[System.Console]::Error.WriteLine("Log directory does not exist: $logDir")
|
||||
exit
|
||||
}
|
||||
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Creates an object the represents the test run.
|
||||
|
||||
.DESCRIPTION
|
||||
build_test_results takes an XML file of results from the CI run,
|
||||
and constructs an object based on that XML file for further
|
||||
processing.
|
||||
|
||||
.OUTPUTS
|
||||
An object with the following elements:
|
||||
assemblyName:
|
||||
assemblyStartDate:
|
||||
assemblyStartTime:
|
||||
assemblyTime:
|
||||
collectionName:
|
||||
collectionTime:
|
||||
allTests: A hashtable with an entry for each port tested
|
||||
The key is the name of the port
|
||||
The value is an object with the following elements:
|
||||
name: Name of the port (Does not include the triplet name)
|
||||
result: Pass/Fail/Skip result from xunit
|
||||
time: Test time in seconds
|
||||
originalResult: Result as defined by Build.h in vcpkg source code
|
||||
abi_tag: The port hash
|
||||
features: The features installed
|
||||
|
||||
.PARAMETER xmlFilename
|
||||
The path to the XML file to parse.
|
||||
#>
|
||||
function build_test_results {
|
||||
[CmdletBinding()]
|
||||
Param
|
||||
(
|
||||
[string]$xmlFilename
|
||||
)
|
||||
if ( ($xmlFilename.Length -eq 0) -or ( -not( Test-Path $xmlFilename))) {
|
||||
#write-error "Missing file: $xmlFilename"
|
||||
return $null
|
||||
}
|
||||
|
||||
Write-Verbose "building test hash for $xmlFilename"
|
||||
|
||||
[xml]$xmlContents = Get-Content $xmlFilename
|
||||
|
||||
# This currently only supports one collection per assembly, which is the way
|
||||
# the vcpkg tests are designed to run in the pipeline.
|
||||
$xmlAssembly = $xmlContents.assemblies.assembly
|
||||
$assemblyName = $xmlAssembly.name
|
||||
$assemblyStartDate = $xmlAssembly."run-date"
|
||||
$assemblyStartTime = $xmlAssembly."run-time"
|
||||
$assemblyTime = $xmlAssembly.time
|
||||
$xmlCollection = $xmlAssembly.collection
|
||||
$collectionName = $xmlCollection.name
|
||||
$collectionTime = $xmlCollection.time
|
||||
|
||||
$allTestResults = @{ }
|
||||
foreach ( $test in $xmlCollection.test) {
|
||||
if (!$test.name.endswith(":$triplet"))
|
||||
{
|
||||
continue
|
||||
}
|
||||
$name = ($test.name -replace ":.*$")
|
||||
|
||||
# Reconstruct the original BuildResult enumeration (defined in Build.h)
|
||||
# failure.message - why the test failed (valid only on test failure)
|
||||
# reason - why the test was skipped (valid only when the test is skipped)
|
||||
# case BuildResult::POST_BUILD_CHECKS_FAILED:
|
||||
# case BuildResult::FILE_CONFLICTS:
|
||||
# case BuildResult::BUILD_FAILED:
|
||||
# case BuildResult::EXCLUDED:
|
||||
# case BuildResult::CASCADED_DUE_TO_MISSING_DEPENDENCIES:
|
||||
$originalResult = "NULLVALUE"
|
||||
switch ($test.result) {
|
||||
"Skip" {
|
||||
$originalResult = $test.reason.InnerText
|
||||
}
|
||||
"Fail" {
|
||||
$originalResult = $test.failure.message.InnerText
|
||||
}
|
||||
"Pass" {
|
||||
$originalResult = "SUCCEEDED"
|
||||
}
|
||||
}
|
||||
|
||||
$abi_tag = ""
|
||||
$features = ""
|
||||
foreach ( $trait in $test.traits.trait) {
|
||||
switch ( $trait.name ) {
|
||||
"abi_tag" { $abi_tag = $trait.value }
|
||||
"features" { $features = $trait.value }
|
||||
}
|
||||
}
|
||||
|
||||
# If additional fields get saved in the XML, then they should be added to this hash
|
||||
# also consider using a PSCustomObject here instead of a hash
|
||||
$testHash = @{ name = $name; result = $test.result; time = $test.time; originalResult = $originalResult; abi_tag = $abi_tag; features = $features }
|
||||
$allTestResults[$name] = $testHash
|
||||
}
|
||||
|
||||
return @{
|
||||
assemblyName = $assemblyName;
|
||||
assemblyStartDate = $assemblyStartDate;
|
||||
assemblyStartTime = $assemblyStartTime;
|
||||
assemblyTime = $assemblyTime;
|
||||
collectionName = $collectionName;
|
||||
collectionTime = $collectionTime;
|
||||
allTests = $allTestResults
|
||||
}
|
||||
}
|
||||
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Creates an object that represents the baseline expectations.
|
||||
|
||||
.DESCRIPTION
|
||||
build_baseline_results converts the baseline file to an object representing
|
||||
the expectations set up by the baseline file. It records four states:
|
||||
1) fail
|
||||
2) skip
|
||||
3) ignore
|
||||
4) pass -- this is represented by not being recorded
|
||||
In other words, if a port is not contained in the object returned by this
|
||||
cmdlet, expect it to pass.
|
||||
|
||||
.OUTPUTS
|
||||
An object containing the following fields:
|
||||
collectionName: the triplet
|
||||
fail: ports marked as fail
|
||||
skip: ports marked as skipped
|
||||
ignore: ports marked as ignore
|
||||
|
||||
.PARAMETER baselineFile
|
||||
The path to vcpkg's ci.baseline.txt.
|
||||
|
||||
.PARAMETER triplet
|
||||
The triplet to create the result object for.
|
||||
#>
|
||||
function build_baseline_results {
|
||||
[CmdletBinding()]
|
||||
Param(
|
||||
$baselineFile,
|
||||
$triplet
|
||||
)
|
||||
#read in the file, strip out comments and blank lines and spaces, leave only the current triplet
|
||||
#remove comments, remove empty lines, remove whitespace, then keep only those lines for $triplet
|
||||
$baseline_list_raw = Get-Content -Path $baselineFile `
|
||||
| Where-Object { -not ($_ -match "\s*#") } `
|
||||
| Where-Object { -not ( $_ -match "^\s*$") } `
|
||||
| ForEach-Object { $_ -replace "\s" } `
|
||||
| Where-Object { $_ -match ":$triplet=" }
|
||||
|
||||
#filter to skipped and trim the triplet
|
||||
$skip_hash = @{ }
|
||||
foreach ( $port in $baseline_list_raw | ? { $_ -match "=skip$" } | % { $_ -replace ":.*$" }) {
|
||||
if ($skip_hash[$port] -ne $null) {
|
||||
[System.Console]::Error.WriteLine("$($port):$($triplet) has multiple definitions in $baselineFile")
|
||||
}
|
||||
$skip_hash[$port] = $true
|
||||
}
|
||||
$fail_hash = @{ }
|
||||
$baseline_list_raw | ? { $_ -match "=fail$" } | % { $_ -replace ":.*$" } | ? { $fail_hash[$_] = $true } | Out-Null
|
||||
$ignore_hash = @{ }
|
||||
$baseline_list_raw | ? { $_ -match "=ignore$" } | % { $_ -replace ":.*$" } | ? { $ignore_hash[$_] = $true } | Out-Null
|
||||
|
||||
return @{
|
||||
collectionName = $triplet;
|
||||
skip = $skip_hash;
|
||||
fail = $fail_hash;
|
||||
ignore = $ignore_hash
|
||||
}
|
||||
}
|
||||
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Analyzes the results of the current run against the baseline.
|
||||
|
||||
.DESCRIPTION
|
||||
combine_results compares the results to the baselie, and generates the results
|
||||
for the CI -- whether it should pass or fail.
|
||||
|
||||
.OUTPUTS
|
||||
An object containing the following:
|
||||
(Note that this is not the same data structure as build_test_results)
|
||||
assemblyName:
|
||||
assemblyStartDate:
|
||||
assemblyStartTime:
|
||||
assemblyTime:
|
||||
collectionName:
|
||||
collectionTime:
|
||||
allTests: A hashtable of each port with a different status from the baseline
|
||||
The key is the name of the port
|
||||
The value is an object with the following data members:
|
||||
name: The name of the port
|
||||
result: xunit test result Pass/Fail/Skip
|
||||
message: Human readable message describing the test result
|
||||
time: time the current test results took to run.
|
||||
baselineResult:
|
||||
currentResult:
|
||||
features:
|
||||
ignored: list of ignored tests
|
||||
|
||||
.PARAMETER baseline
|
||||
The baseline object to use from build_baseline_results.
|
||||
|
||||
.PARAMETER current
|
||||
The results object to use from build_test_results.
|
||||
#>
|
||||
function combine_results {
|
||||
[CmdletBinding()]
|
||||
Param
|
||||
(
|
||||
$baseline,
|
||||
$current
|
||||
)
|
||||
|
||||
if ($baseline.collectionName -ne $current.collectionName) {
|
||||
Write-Warning "Comparing mismatched collections $($baseline.collectionName) and $($current.collectionName)"
|
||||
}
|
||||
|
||||
$currentTests = $current.allTests
|
||||
|
||||
# lookup table with the results of all of the tests
|
||||
$allTestResults = @{ }
|
||||
|
||||
$ignoredList = @()
|
||||
|
||||
Write-Verbose "analyzing $($currentTests.count) tests"
|
||||
|
||||
foreach ($key in $currentTests.keys) {
|
||||
Write-Verbose "analyzing $key"
|
||||
|
||||
$message = $null
|
||||
$result = $null
|
||||
$time = $null
|
||||
$currentResult = $null
|
||||
$features = $currentTest.features
|
||||
|
||||
$baselineResult = "Pass"
|
||||
if ($baseline.fail[$key] -ne $null) {
|
||||
Write-Verbose "$key is failing"
|
||||
$baselineResult = "Fail"
|
||||
}
|
||||
elseif ( $baseline.skip[$key] -ne $null) {
|
||||
Write-Verbose "$key is skipped"
|
||||
$baselineResult = "Skip"
|
||||
}
|
||||
elseif ( $baseline.ignore[$key] -ne $null) {
|
||||
$baselineResult = "ignore"
|
||||
}
|
||||
|
||||
$currentTest = $currentTests[$key]
|
||||
|
||||
if ( $currentTest.result -eq $baselineResult) {
|
||||
Write-Verbose "$key has no change from baseline"
|
||||
$currentResult = $currentTest.result
|
||||
if ($allResults) {
|
||||
# Only marking regressions as failures but keep the skipped status
|
||||
if ($currentResult -eq "Skip") {
|
||||
$result = "Skip"
|
||||
}
|
||||
else {
|
||||
$result = "Pass"
|
||||
}
|
||||
$message = "No change from baseline"
|
||||
$time = $currentTest.time
|
||||
}
|
||||
}
|
||||
elseif ( $baselineResult -eq "ignore") {
|
||||
if ( $currentTest.result -eq "Fail" ) {
|
||||
Write-Verbose "ignoring failure on $key"
|
||||
$ignoredList += $key
|
||||
}
|
||||
}
|
||||
else {
|
||||
Write-Verbose "$key had a change from the baseline"
|
||||
|
||||
$currentResult = $currentTest.result
|
||||
# Test exists in both test runs but does not match. Determine if this is a regression
|
||||
# Pass -> Fail = Fail (Regression)
|
||||
# Pass -> Skip = Skip
|
||||
# Fail -> Pass = Fail (need to update baseline)
|
||||
# Fail -> Skip = Skip
|
||||
# Skip -> Fail = Fail (Should not happen)
|
||||
# Skip -> Pass = Fail (should not happen)
|
||||
|
||||
$lookupTable = @{
|
||||
'Pass' = @{
|
||||
'Fail' = @('Fail', "Test passes in baseline but fails in current run. If expected update ci.baseline.txt with '$($key):$($current.collectionName)=fail'");
|
||||
'Skip' = @($null, 'Test was skipped due to missing dependencies')
|
||||
};
|
||||
'Fail' = @{
|
||||
'Pass' = @('Fail', "Test fails in baseline but now passes. Update ci.baseline.txt with '$($key):$($current.collectionName)=pass'");
|
||||
'Skip' = @($null, 'Test fails in baseline but is skipped in current run')
|
||||
};
|
||||
'Skip' = @{
|
||||
'Fail' = @('Skip', "Test is skipped in baseline but fails in current run. Results are ignored")
|
||||
'Pass' = @('Skip', "Test is skipped in baseline but passes in current run. Results are ignored")
|
||||
}
|
||||
}
|
||||
$resultList = $lookupTable[$baselineResult][$currentResult]
|
||||
$result = $resultList[0]
|
||||
$message = $resultList[1]
|
||||
$time = $currentTest.time
|
||||
Write-Verbose ">$key $message"
|
||||
}
|
||||
|
||||
if ($result -ne $null) {
|
||||
Write-Verbose "Adding $key to result list"
|
||||
$allTestResults[$key] = @{ name = $key; result = $result; message = $message; time = $time; abi_tag = $currentTest.abi_tag; baselineResult = $baselineResult; currentResult = $currentResult; features = $features }
|
||||
}
|
||||
}
|
||||
|
||||
return @{
|
||||
assemblyName = $current.assemblyName;
|
||||
assemblyStartDate = $current.assemblyStartDate;
|
||||
assemblyStartTime = $current.assemblyStartTime;
|
||||
assemblyTime = $current.assemblyTime;
|
||||
collectionName = $current.collectionName;
|
||||
collectionTime = $current.collectionTime;
|
||||
allTests = $allTestResults;
|
||||
ignored = $ignoredList
|
||||
}
|
||||
}
|
||||
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Writes short errors to the CI logs.
|
||||
|
||||
.DESCRIPTION
|
||||
write_errors_for_summary takes a hashtable from triplets to combine_results
|
||||
objects, and writes short errors to the CI logs.
|
||||
|
||||
.PARAMETER complete_results
|
||||
A hashtable from triplets to combine_results objects.
|
||||
#>
|
||||
function write_errors_for_summary {
|
||||
[CmdletBinding()]
|
||||
Param(
|
||||
$complete_results
|
||||
)
|
||||
|
||||
$failure_found = $false
|
||||
|
||||
Write-Verbose "preparing error output for Azure Devops"
|
||||
|
||||
foreach ($triplet in $complete_results.Keys) {
|
||||
$triplet_results = $complete_results[$triplet]
|
||||
|
||||
Write-Verbose "searching $triplet triplet"
|
||||
|
||||
# add each port results
|
||||
foreach ($testName in $triplet_results.allTests.Keys) {
|
||||
$test = $triplet_results.allTests[$testName]
|
||||
|
||||
Write-Verbose "checking $($testName):$triplet $($test.result)"
|
||||
|
||||
if ($test.result -eq 'Fail') {
|
||||
if (($test.currentResult) -eq "pass" -and $passingIsPassing) {
|
||||
continue;
|
||||
}
|
||||
|
||||
$failure_found = $true
|
||||
if ($test.currentResult -eq "pass") {
|
||||
[System.Console]::Error.WriteLine( `
|
||||
"PASSING, REMOVE FROM FAIL LIST: $($test.name):$triplet ($baselineFile)" `
|
||||
)
|
||||
}
|
||||
else {
|
||||
[System.Console]::Error.WriteLine( `
|
||||
"REGRESSION: $($test.name):$triplet. If expected, add $($test.name):$triplet=fail to $baselineFile." `
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
$complete_results = @{ }
|
||||
Write-Verbose "looking for $triplet logs"
|
||||
|
||||
# The standard name for logs is:
|
||||
# <triplet>.xml
|
||||
# for example:
|
||||
# x64-linux.xml
|
||||
|
||||
$current_test_hash = build_test_results( Convert-Path "$logDir\$($triplet).xml" )
|
||||
$baseline_results = build_baseline_results -baselineFile $baselineFile -triplet $triplet
|
||||
|
||||
if ($current_test_hash -eq $null) {
|
||||
[System.Console]::Error.WriteLine("Missing $triplet test results in current test run")
|
||||
$missing_triplets[$triplet] = "test"
|
||||
}
|
||||
else {
|
||||
Write-Verbose "combining results..."
|
||||
$complete_results[$triplet] = combine_results -baseline $baseline_results -current $current_test_hash
|
||||
}
|
||||
|
||||
Write-Verbose "done analyzing results"
|
||||
|
||||
# emit error last. Unlike the table output this is going to be seen in the "status" section of the pipeline
|
||||
# and needs to be formatted for a single line.
|
||||
write_errors_for_summary -complete_results $complete_results
|
||||
@ -1,83 +0,0 @@
|
||||
# Copyright (c) Microsoft Corporation.
|
||||
# SPDX-License-Identifier: MIT
|
||||
#
|
||||
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Generates a list of ports to skip in the CI.
|
||||
|
||||
.DESCRIPTION
|
||||
generate-skip-list takes a triplet, and the path to the ci.baseline.txt
|
||||
file, and generates a skip list string to pass to vcpkg.
|
||||
|
||||
.PARAMETER Triplet
|
||||
The triplet to find skipped ports for.
|
||||
|
||||
.PARAMETER BaselineFile
|
||||
The path to the ci.baseline.txt file.
|
||||
#>
|
||||
[CmdletBinding()]
|
||||
Param(
|
||||
[string]$Triplet,
|
||||
[string]$BaselineFile,
|
||||
[switch]$SkipFailures = $false
|
||||
)
|
||||
|
||||
$ErrorActionPreference = 'Stop'
|
||||
|
||||
if (-not (Test-Path -Path $BaselineFile)) {
|
||||
Write-Error "Unable to find baseline file $BaselineFile"
|
||||
throw
|
||||
}
|
||||
|
||||
#read in the file, strip out comments and blank lines and spaces
|
||||
$baselineListRaw = Get-Content -Path $BaselineFile `
|
||||
| Where-Object { -not ($_ -match "\s*#") } `
|
||||
| Where-Object { -not ( $_ -match "^\s*$") } `
|
||||
| ForEach-Object { $_ -replace "\s" }
|
||||
|
||||
###############################################################
|
||||
# This script is running at the beginning of the CI test, so do a little extra
|
||||
# checking so things can fail early.
|
||||
|
||||
#verify everything has a valid value
|
||||
$missingValues = $baselineListRaw | Where-Object { -not ($_ -match "=\w") }
|
||||
|
||||
if ($missingValues) {
|
||||
Write-Error "The following are missing values: $missingValues"
|
||||
throw
|
||||
}
|
||||
|
||||
$invalidValues = $baselineListRaw `
|
||||
| Where-Object { -not ($_ -match "=(skip|pass|fail|ignore)$") }
|
||||
|
||||
if ($invalidValues) {
|
||||
Write-Error "The following have invalid values: $invalidValues"
|
||||
throw
|
||||
}
|
||||
|
||||
$baselineForTriplet = $baselineListRaw `
|
||||
| Where-Object { $_ -match ":$Triplet=" }
|
||||
|
||||
# Verify there are no duplicates (redefinitions are not allowed)
|
||||
$file_map = @{ }
|
||||
foreach ($port in $baselineForTriplet | ForEach-Object { $_ -replace ":.*$" }) {
|
||||
if ($null -ne $file_map[$port]) {
|
||||
Write-Error `
|
||||
"$($port):$($Triplet) has multiple definitions in $baselineFile"
|
||||
throw
|
||||
}
|
||||
$file_map[$port] = $true
|
||||
}
|
||||
|
||||
# Format the skip list for the command line
|
||||
if ($SkipFailures) {
|
||||
$targetRegex = "=(?:skip|fail)$"
|
||||
} else {
|
||||
$targetRegex = "=skip$"
|
||||
}
|
||||
|
||||
$skip_list = $baselineForTriplet `
|
||||
| Where-Object { $_ -match $targetRegex } `
|
||||
| ForEach-Object { $_ -replace ":.*$" }
|
||||
[string]::Join(",", $skip_list)
|
||||
@ -136,19 +136,6 @@ if ($LASTEXITCODE -ne 0)
|
||||
throw "vcpkg clean failed"
|
||||
}
|
||||
|
||||
$skipList = . "$PSScriptRoot/generate-skip-list.ps1" `
|
||||
-Triplet $Triplet `
|
||||
-BaselineFile "$PSScriptRoot/../ci.baseline.txt" `
|
||||
-SkipFailures:$skipFailures
|
||||
|
||||
$hostArgs = @()
|
||||
if ($Triplet -in @('x64-windows', 'x64-osx', 'x64-linux'))
|
||||
{
|
||||
# WORKAROUND: These triplets are native-targetting which triggers an issue in how vcpkg handles the skip list.
|
||||
# The workaround is to pass the skip list as host-excludes as well.
|
||||
$hostArgs = @("--host-exclude=$skipList")
|
||||
}
|
||||
|
||||
$parentHashes = @()
|
||||
if (($BuildReason -eq 'PullRequest') -and -not $NoParentHashes)
|
||||
{
|
||||
@ -169,7 +156,7 @@ if (($BuildReason -eq 'PullRequest') -and -not $NoParentHashes)
|
||||
# but changes must trigger at least some testing.
|
||||
Copy-Item "scripts/buildsystems/vcpkg.cmake" -Destination "scripts/test_ports/cmake"
|
||||
Copy-Item "scripts/buildsystems/vcpkg.cmake" -Destination "scripts/test_ports/cmake-user"
|
||||
& "./vcpkg$executableExtension" ci $Triplet --dry-run --exclude=$skipList @hostArgs @commonArgs --no-binarycaching "--output-hashes=$parentHashesFile"
|
||||
& "./vcpkg$executableExtension" ci "--triplet=$Triplet" --dry-run "--ci-baseline=$PSScriptRoot/../ci.baseline.txt" @commonArgs --no-binarycaching "--output-hashes=$parentHashesFile"
|
||||
|
||||
Write-Host "Running CI using parent hashes"
|
||||
& git reset --hard HEAD
|
||||
@ -179,7 +166,7 @@ if (($BuildReason -eq 'PullRequest') -and -not $NoParentHashes)
|
||||
# but changes must trigger at least some testing.
|
||||
Copy-Item "scripts/buildsystems/vcpkg.cmake" -Destination "scripts/test_ports/cmake"
|
||||
Copy-Item "scripts/buildsystems/vcpkg.cmake" -Destination "scripts/test_ports/cmake-user"
|
||||
& "./vcpkg$executableExtension" ci $Triplet --x-xunit=$xmlFile --exclude=$skipList --failure-logs=$failureLogs @hostArgs @commonArgs @cachingArgs @parentHashes
|
||||
& "./vcpkg$executableExtension" ci "--triplet=$Triplet" --x-xunit=$xmlFile --failure-logs=$failureLogs "--ci-baseline=$PSScriptRoot/../ci.baseline.txt" @commonArgs @cachingArgs @parentHashes
|
||||
|
||||
$failureLogsEmpty = (-Not (Test-Path $failureLogs) -Or ((Get-ChildItem $failureLogs).count -eq 0))
|
||||
Write-Host "##vso[task.setvariable variable=FAILURE_LOGS_EMPTY]$failureLogsEmpty"
|
||||
@ -188,8 +175,3 @@ if ($LASTEXITCODE -ne 0)
|
||||
{
|
||||
throw "vcpkg ci failed"
|
||||
}
|
||||
|
||||
& "$PSScriptRoot/analyze-test-results.ps1" -logDir $xmlResults `
|
||||
-triplet $Triplet `
|
||||
-baselineFile .\scripts\ci.baseline.txt `
|
||||
-passingIsPassing:$PassingIsPassing
|
||||
|
||||
@ -45,7 +45,7 @@ while (!($vcpkgRootDir -eq "") -and !(Test-Path "$vcpkgRootDir\.vcpkg-root"))
|
||||
|
||||
Write-Verbose "Examining $vcpkgRootDir for .vcpkg-root - Found"
|
||||
|
||||
$versionDate = '2022-03-09'
|
||||
$versionDate = '2022-03-25'
|
||||
if ($env:PROCESSOR_ARCHITECTURE -eq 'ARM64' -or $env:PROCESSOR_IDENTIFIER -match "ARMv[8,9] \(64-bit\)") {
|
||||
& "$scriptsDir/tls12-download.exe" github.com "/microsoft/vcpkg-tool/releases/download/$versionDate/vcpkg-arm64.exe" "$vcpkgRootDir\vcpkg.exe"
|
||||
} else {
|
||||
|
||||
@ -126,23 +126,23 @@ fi
|
||||
|
||||
# Choose the vcpkg binary to download
|
||||
vcpkgDownloadTool="ON"
|
||||
vcpkgToolReleaseTag="2022-03-09"
|
||||
vcpkgToolReleaseTag="2022-03-25"
|
||||
if [ "$UNAME" = "Darwin" ]; then
|
||||
echo "Downloading vcpkg-macos..."
|
||||
vcpkgToolReleaseSha="fc04f2db2676e9b5958c3bf2c8537f470acb1ed9e0a2b6be1145feebbb701251644dc73311fe5cc4d791272b90e3810d358ded6ca9436700a531634d405180c2"
|
||||
vcpkgToolReleaseSha="a48a35949362182a792124a6088a2ac24e1992e4ef2604ae369d44b2b7745a422dd78f74cba5807f52312674ea7bcd11d79435faffb8391b3703bc43c0d95605"
|
||||
vcpkgToolName="vcpkg-macos"
|
||||
elif [ "$vcpkgUseMuslC" = "ON" ]; then
|
||||
echo "Downloading vcpkg-muslc..."
|
||||
vcpkgToolReleaseSha="0d9dc5669a389601d919893e9e742ec41e5ee09c7aaea7d2f89e8f388f6e08296dea4132f016e44dac771c8b40aabcf195dd228ade8cd0a199cde037d8fdce49"
|
||||
vcpkgToolReleaseSha="d0995018de0b20188291aeabafff5296ab33ef334b67c7512f3cd511dfc1740af62d8ad863e48df5731a6d44e2fbdfe060410494449d61743bd767e87fe68b20"
|
||||
vcpkgToolName="vcpkg-muslc"
|
||||
elif [ "$ARCH" = "x86_64" ]; then
|
||||
echo "Downloading vcpkg-glibc..."
|
||||
vcpkgToolReleaseSha="1bc60ca03d76d75033d3cc864e96ff8f36843661c744e6bb7554ee50d5bd8dd027d5b7cdc7909801e75a72b44ac04d66dac29bb6669569c65c62f06f0064dd88"
|
||||
vcpkgToolReleaseSha="c66e74d4469849c063e9e165b034a529e61e8245fd608a9bfb6c5e25a5a530951ea6995f5b1ee20aac1632f094f49ba3067516506dd2a81122a686ff5826efd9"
|
||||
vcpkgToolName="vcpkg-glibc"
|
||||
else
|
||||
echo "Unable to determine a binary release of vcpkg; attempting to build from source."
|
||||
vcpkgDownloadTool="OFF"
|
||||
vcpkgToolReleaseSha="409480568f161055ba1bb41fc6e9ed99e196f180731ee05422ab4ba182d6277b5a59048b1b10fe39e10c8f4cc8becd2f752fabb775b4623e6888a00af7af782f"
|
||||
vcpkgToolReleaseSha="15188f9c7370b308d54df3a47bba56d7cbcf437252c3c4545b8aaa7e459e06c47cbdf3d3a8d98bd1ef0d39ccbf2e70b4c0a6d99ce0a58db866b95606d46f92d6"
|
||||
fi
|
||||
|
||||
# Do the download or build.
|
||||
|
||||
@ -118,23 +118,27 @@ charls:x64-windows-static=skip
|
||||
charls:x64-windows-static-md=skip
|
||||
charls:x86-windows=skip
|
||||
# chartdir does not offer stable download URLs
|
||||
chartdir:arm64-windows=fail
|
||||
chartdir:arm-uwp=fail
|
||||
chartdir:x64-uwp=fail
|
||||
chartdir:x64-windows-static=fail
|
||||
chartdir:x86-windows=fail
|
||||
chartdir:x64-windows=fail
|
||||
chartdir:x64-windows-static-md=fail
|
||||
chartdir:x64-osx=fail
|
||||
chartdir:arm64-windows=skip
|
||||
chartdir:arm-uwp=skip
|
||||
chartdir:x64-uwp=skip
|
||||
chartdir:x64-windows-static=skip
|
||||
chartdir:x86-windows=skip
|
||||
chartdir:x64-windows=skip
|
||||
chartdir:x64-windows-static-md=skip
|
||||
chartdir:x64-osx=skip
|
||||
chmlib:arm-uwp=fail
|
||||
chmlib:x64-uwp=fail
|
||||
|
||||
# chromium-base does not yet support VS2022
|
||||
chromium-base:x64-windows=fail
|
||||
chromium-base:x64-windows-static=fail
|
||||
|
||||
# Chromium Base requires a recent version of Clang to be installed.
|
||||
# chromium-base has several problems and is upgraded to "skip" because it hits a lot of servers that can slow CI
|
||||
# broken on Windows because it does not yet support VS2022
|
||||
chromium-base:x64-windows=skip
|
||||
chromium-base:x64-windows-static=skip
|
||||
chromium-base:x64-windows-static-md=skip
|
||||
# broken on Linux because
|
||||
chromium-base:x64-linux=skip
|
||||
# broken on OSX for unknown reasons; it appears to be messing with some form of system-wide cache
|
||||
# because the first attempt to build it fails, but subsequent attempts succeed
|
||||
chromium-base:x64-osx=skip
|
||||
|
||||
civetweb:arm64-windows = skip
|
||||
civetweb:arm-uwp = skip
|
||||
@ -159,6 +163,7 @@ coin:x64-uwp=fail
|
||||
colmap:x86-windows=fail
|
||||
colmap:x64-windows=fail
|
||||
colmap:x64-windows-static=fail
|
||||
colmap:x64-windows-static-md=fail
|
||||
|
||||
concurrencpp:x64-linux=fail
|
||||
constexpr-contracts:x64-linux=fail
|
||||
@ -360,9 +365,6 @@ gmmlib:x64-windows-static=fail
|
||||
gmmlib:x64-windows-static-md=fail
|
||||
gmmlib:x86-windows=fail
|
||||
|
||||
# the msbuild for gmp:x64-uwp is broken on VS2022 due to TargetPlatformMinVersion not existing
|
||||
gmp:x64-uwp=fail
|
||||
|
||||
google-cloud-cpp:arm-uwp=fail
|
||||
google-cloud-cpp:x64-uwp=fail
|
||||
gppanel:x64-osx=fail
|
||||
@ -513,8 +515,9 @@ libgo:arm-uwp=fail
|
||||
libgo:x64-uwp=fail
|
||||
libgo:arm64-windows=fail
|
||||
|
||||
# the msbuild for libgpg:x64-uwp is broken on VS2022 due to TargetPlatformMinVersion not existing
|
||||
# the msbuild for libgpg:x64-uwp and libgpg-error:x64-uwp are broken on VS2022 due to TargetPlatformMinVersion not existing
|
||||
libgpg:x64-uwp=fail
|
||||
libgpg-error:x64-uwp=fail
|
||||
|
||||
libhdfs3:x64-linux=fail
|
||||
libhdfs3:x64-osx=fail
|
||||
@ -558,6 +561,7 @@ libmesh:x64-windows-static-md=skip
|
||||
libmesh:x86-windows=skip
|
||||
libmesh:x64-osx=skip
|
||||
libmesh:x64-linux=skip
|
||||
libmikmod:x64-osx=fail
|
||||
libmodman:arm-uwp=fail
|
||||
libmodman:x64-uwp=fail
|
||||
libmodman:x64-windows-static=fail
|
||||
@ -1404,10 +1408,8 @@ lapack-reference:x64-uwp=skip
|
||||
# failures for x64-windows-static-md
|
||||
ace:x64-windows-static-md=fail
|
||||
akali:x64-windows-static-md=fail
|
||||
chromium-base:x64-windows-static-md=fail
|
||||
clockutils:x64-windows-static-md=fail
|
||||
fastcgi:x64-windows-static-md=fail
|
||||
gmp:x64-windows-static-md=fail
|
||||
ijg-libjpeg:x64-windows-static-md=fail
|
||||
libcerf:x64-windows-static-md=fail
|
||||
libmicrohttpd:x64-windows-static-md=fail
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user