39 Commits

Author SHA1 Message Date
Billy O'Neal
ab00a636c7
Update Windows Fleet for December 2023 Patch Tuesday (#35640) 2023-12-18 10:27:45 -08:00
dan-shaw
5787cfa699
[vcpkg] Android CI support (#29406)
* initial

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix
2023-04-24 21:22:59 -07:00
Kai Pastor
0746c2aab7
Track CI baseline change effect (#30756)
* Test ci baseline change effect

* Skip parent hashes when baseline was modified

* Update head baseline variable

* Revert "Test ci baseline change effect"

This reverts commit a5e7a5c404eb400476555b92e51b3f5a87880300.

* Read raw data
2023-04-11 23:11:59 -07:00
Billy O'Neal
0abb2b2d88
Update vcpkg-tool to 2022-06-15 (#25226)
* Update vcpkg-tool to 2022-06-15. See https://github.com/microsoft/vcpkg-tool/releases/tag/2022-06-15
* Add `=pass` entries from https://github.com/microsoft/vcpkg-tool/pull/579
* Cherry pick XUnit output from https://github.com/microsoft/vcpkg/pull/23477 which hooks up https://github.com/microsoft/vcpkg-tool/pull/585 (Thanks again @dg0yt !)
* Update release process document now that we can do the full tree rebuild to test the tool release before signing.

Thank you:

@autoantwort 
@christophe-calmejane 
@JackBoosY 
@benmcmorran
@hmoffatt 
@klalumiere 
@Thomas1664 
@Neumann-A 
@AenBleidd

for your contributions!

Fixes https://github.com/microsoft/vcpkg/issues/22686 .
Fixes https://github.com/microsoft/vcpkg/issues/21800 .
Fixes https://github.com/microsoft/vcpkg/issues/16216 .
Fixes https://github.com/microsoft/vcpkg/issues/15836 .

Full tree rebuild testing this update is: https://dev.azure.com/vcpkg/public/_build/results?buildId=73664&view=results
2022-06-15 11:20:52 -07:00
Billy O'Neal
14e7bb4ae2
Update vcpkg-tool to 2022-05-05 (#24526)
* Update vcpkg-tool to 2022-05-03

* Remove unused --x-xunit call.

* Actually 2022-05-05
2022-05-10 15:54:52 -07:00
nicole mazzuca
6c617b9f55
[vcpkg-tool] update to 2022-03-30 (#23886)
* [vcpkg-tool] update to 2022-03-30

* add --allow-unexpected-passing, --skip-failures

* Robert CR

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
2022-03-31 19:08:09 -07:00
Billy O'Neal
671db8ae9d
Update vcpkg-tool to 2022-03-25 (#23757)
* Update vcpkg-tool to 2022-03-24

* Hook up https://github.com/microsoft/vcpkg-tool/pull/345

* Hook up https://github.com/microsoft/vcpkg-tool/pull/442

* Update vcpkg-tool to 2022-03-25

* Analysis of failures.

* [Most recent nightly build failed](https://dev.azure.com/vcpkg/public/_build/results?buildId=69427)
* [Validation of this tool update failed](https://dev.azure.com/vcpkg/public/_build/results?buildId=69417)

## Common to both:

PASSING, REMOVE FROM FAIL LIST: chartdir:x64-windows (.\scripts\ci.baseline.txt)
PASSING, REMOVE FROM FAIL LIST: chartdir:x64-windows-static-md (.\scripts\ci.baseline.txt)
PASSING, REMOVE FROM FAIL LIST: chartdir:x86-windows (.\scripts\ci.baseline.txt)

Probably fixed by https://github.com/microsoft/vcpkg/pull/23701

PASSING, REMOVE FROM FAIL LIST: gmp:x64-uwp (.\scripts\ci.baseline.txt)
PASSING, REMOVE FROM FAIL LIST: gmp:x64-windows-static-md (.\scripts\ci.baseline.txt)

Probably fixed by https://github.com/microsoft/vcpkg/pull/23466 ?

REGRESSION: colmap:x64-windows-static-md failed with BUILD_FAILED. If expected, add colmap:x64-windows-static-md=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

I don't know exactly what changed. I observe that
* this thing depends on a *lot* of stuff
* on March 14 we didn't even attempt to build this
* the x64-windows ones are already in the baseline

so I skipped it.

REGRESSION: qtdeclarative:x64-windows. If expected, add qtdeclarative:x64-windows=fail to .\scripts\ci.baseline.txt.
REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\1\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: qtdeclarative:x64-windows failed with BUILD_FAILED. If expected, add qtdeclarative:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

This is a reporting change: The new world order also includes host build failures which is why it's duplicated.

See also https://github.com/microsoft/vcpkg/issues/23714
See also https://github.com/microsoft/vcpkg/issues/23490

I'm nervous about baslining this because it seems most of the qt world is built on top of this port

I filed https://github.com/microsoft/vcpkg/issues/23824 about this and @Neumann-A indicated this should be fixed by https://github.com/microsoft/vcpkg/pull/23755

REGRESSION: nettle:x64-uwp. If expected, add nettle:x64-uwp=fail to .\scripts\ci.baseline.txt.
REGRESSION: nettle:x64-windows-static-md. If expected, add nettle:x64-windows-static-md=fail to .\scripts\ci.baseline.txt.
REGRESSION: nettle:x64-uwp failed with BUILD_FAILED. If expected, add nettle:x64-uwp=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.
REGRESSION: nettle:x64-windows-static-md failed with POST_BUILD_CHECKS_FAILED. If expected, add nettle:x64-windows-static-md=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

Didn't analyze, probably fixed by https://github.com/microsoft/vcpkg/pull/23519 ?

REGRESSION: libgpg-error:x64-uwp. If expected, add libgpg-error:x64-uwp=fail to .\scripts\ci.baseline.txt.
REGRESSION: libgpg-error:x64-uwp failed with BUILD_FAILED. If expected, add libgpg-error:x64-uwp=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

This was broken by VS2022 update:
```
C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Microsoft\VisualStudio\v17.0\AppxPackage\Microsoft.AppXPackage.Targets(892,25): error MSB4086: A numeric comparison was attempted on "$(TargetPlatformMinVersion)" that evaluates to "" instead of a number, in condition "'$(TargetPlatformMinVersion)' >= '10.0.17200.0'". [C:\Dev\vcpkg\buildtrees\libgpg-error\x64-uwp-rel\error-1.42-2324ddbc71.clean\SMP\libgpg-error_winrt.vcxproj]
```

REGRESSION: libmikmod:x64-osx. If expected, add libmikmod:x64-osx=fail to .\scripts\ci.baseline.txt.
REGRESSION: libmikmod:x64-osx failed with BUILD_FAILED. If expected, add libmikmod:x64-osx=fail to /Users/vagrant/Data/work/2/s/scripts/azure-pipelines/../ci.baseline.txt.

Broken between [2022-03-16](https://dev.azure.com/vcpkg/public/_build/results?buildId=68947) and [2022-03-18](https://dev.azure.com/vcpkg/public/_build/results?buildId=69051). Unfortunately I don't see obvious reasons why. Nothing else depends on this and nobody has noticed in 2 weeks, so I'm baslining it for now. (Will investigate shortly...)

## Only broken in tool update:

REGRESSION: mesa:x64-windows failed with BUILD_FAILED. If expected, add mesa:x64-windows=fail to C:\a\2\s\scripts\azure-pipelines/../ci.baseline.txt.

```
-- Downloading https://gitlab.freedesktop.org/mesa/mesa/-/archive/mesa-21.2.5/mesa-mesa-21.2.5.tar.gz -> mesa-mesa-mesa-21.2.5-1.tar.gz...
-- Extracting source /Users/vagrant/Data/downloads/mesa-mesa-mesa-21.2.5-1.tar.gz
-- Applying patch swravx512-post-static-link.patch
-- Applying patch swr-msvc-2.patch
-- Applying patch swr-llvm13.patch
-- Applying patch radv-msvc-llvm13-2.patch
-- Applying patch d3d10sw.patch
-- Using source at /Users/vagrant/Data/buildtrees/mesa/src/esa-21.2.5-2df234d2b1.clean
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'mako'
CMake Error at ports/mesa/portfile.cmake:85 (message):
  Python package 'mako' needs to be installed for port 'mesa'.

  Complete list of required python packages: setuptools;mako
Call Stack (most recent call first):
  ports/mesa/portfile.cmake:91 (vcpkg_get_python_package)
  scripts/ports.cmake:145 (include)
```

Looks like this is being tracked by https://github.com/microsoft/vcpkg/pull/23089 ; perhaps that we don't have as aggressive a recycling strategy for macos boxes as we do for the others has let different machines give different results?

## Only broken without tool update:

REGRESSION: chromium-base:x64-osx. If expected, add chromium-base:x64-osx=fail to .\scripts\ci.baseline.txt.

This one has been constantly flaky; I baselined it.

REGRESSION: libxml2:x64-osx. If expected, add libxml2:x64-osx=fail to .\scripts\ci.baseline.txt.

This port uses vcpkg_from_git and the upstream server was down during the build.

* Restore chartdir to the baseline, I thought https://github.com/microsoft/vcpkg/pull/23732 had been merged.
2022-03-28 13:17:35 -07:00
Kai Pastor
7310241d5d
[vcpk-ci] Trigger some test ports from vcpkg.cmake changes (#23430)
* Trigger some test ports from vcpkg.cmake changes

* Bump test port version

* Revert "Bump test port version"

* Use old toolchain version for parent hashes

* Add ABI hashing note to vcpkg.cmake

* Test vcpkg.cmake with cmake instead vpckg-ci-paraview
2022-03-23 16:16:17 -07:00
nicole mazzuca
5a8a1dfff8
[ci] create parent-hashes in ArtifactStagingDirectory (#23468)
instead of WorkingRoot - this should mean that we don't get
permission errors in CI

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
2022-03-09 14:29:15 -08:00
Billy O'Neal
be5c4ef68b
Update vcpkg-tool to 2022-03-08. (#23424)
* Update vcpkg-tool to 2022-03-08.

* Back off required git version to something actually in the build lab.

* Remove hiding of the output of --parent-hashes.

* Update 7zip reference as indicated in https://github.com/microsoft/vcpkg-tool/pull/413
2022-03-08 12:47:23 -08:00
Kai Pastor
61a704cfd6
[vcpkg-ci] Modify task names (#23361)
* Add triplet to name of main CI task

* Fix osx bootstrap task name

* Hide output from switch to parent hash

* Bump test port version

* Hardcode osx+linux triplet names

* Improve logging order

* Add error handling to each tool download

* Revert "Bump test port version"

* Don't prefetch 7zip
2022-03-07 11:39:32 -08:00
Kai Pastor
43cb9a9026
Explicitly fetch 7zip (#23302) 2022-02-28 09:33:52 -08:00
Billy O'Neal
d70768e374
[vcpkg] Change osx to run every 48 hours rather than every 24. (#21954)
* Change osx to run every 48 hours rather than every 24.

* Back out more former-tensorflow-separation yaml.
2021-12-13 12:54:16 -08:00
Kai Pastor
eea02380aa
[ci] Remove unaffected ports from pull request builds (#21078)
* Rewrite to single invovation of 'vcpkg ci'

* Split caching args from common args

* Use parent hashes for pull request CI builds

* Completely switch to parent revision state and back
2021-11-01 12:18:28 -07:00
Billy O'Neal
4474aba1e7
[ci] Attempt to make CI more reliable by using robocopy to delete (#20889) 2021-10-20 22:51:29 -07:00
Billy O'Neal
078f3e51ef
[ci] Skip attempting to upload failure logs if there is no failure logs directory. (#20826) 2021-10-18 15:04:49 -07:00
Billy O'Neal
f670aa5657
Use the correct SAS token for macos and fix tensorflow build failing on success. (#20512)
* Use the correct SAS token for macos and fix tensorflow build failing on success.

* Fix binary cache.
2021-10-04 11:52:47 -07:00
nicole mazzuca
4187499295
[macos ci] oops that was a dumb bug (#20448) 2021-09-30 19:52:23 -07:00
nicole mazzuca
45315bc3aa
[macos ci] move out tensorflow (#20388)
* [macos ci] move out tensorflow

* rename ExtraPorts and CorePorts

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
Co-authored-by: Billy Robert O'Neal III <bion@microsoft.com>
2021-09-29 13:15:25 -07:00
Kai Pastor
827a54c404
[ci] Don't emit warnings for absence of failure logs (#20316)
* No warning for absence of failure logs

* Update condition for osx and linux

Co-authored-by: Billy Robert O'Neal III <bion@microsoft.com>
2021-09-28 14:00:33 -07:00
Billy O'Neal
93a8320399
Back out "cascades" enforcement because it caused too many stealth merge conflicts. (#20181)
Mostly reverts https://github.com/microsoft/vcpkg/pull/20140

See https://github.com/microsoft/vcpkg/discussions/20180 for further discussion.
2021-09-15 11:39:39 -07:00
Billy O'Neal
5395ad9a4f
Add "cascades" enforcement (#20140)
* Add "cascades" enforcement as requested by https://github.com/microsoft/vcpkg/pull/20074#issuecomment-916565430

* Add counts.

* Fix arm-uwp count.

* Update counts.

* Update counts
2021-09-14 17:46:38 -07:00
Billy O'Neal
f78c96f706
Decouple the storage account used for binary caching from the images. (#19912)
* Decouple the storage account used for binary caching from the images.

We are being forced to change infrastructure somewhat more frequently than we would like, so change the binary caches to use a shared storage account like we already use for asset caching.

* blob blob blob

* Add quotes

* Avoid name stomping.
2021-09-03 00:02:29 -07:00
Billy O'Neal
b67fab9861
Do not emit 'PASSING, REMOVE FROM FAIL LIST' results when using vcpkg to test prerelease versions of MSVC++ (#19011) 2021-07-19 19:55:00 -07:00
Phoebe
117cf538f4
[vcpkg] Remove workaround in Ci pipeline (#18119) 2021-05-26 13:55:22 -07:00
Robert Schumacher
4e83da0887
[vcpkg] Update vcpkg tool version 2021-05-05 (#17689)
* [vcpkg] Update vcpkg tool version 2021-05-05

* [vcpkg] Hotfix bug in Microsoft/vcpkg-tool#34

* [vcpkg-ci] Ignore results from non-target triplets

* [protobuf] Fix regression introduced in #16904

* [vcpkg] Fix bug where 'staticcrt' was ignored

Co-authored-by: Robert Schumacher <ras0219@outlook.com>
2021-05-06 07:12:43 -07:00
ras0219
b5bb1511f0
[many ports] Apply host dependencies (#16479)
* [vcpkg] Add VCPKG_HOST_TRIPLET. Improve vcpkg.schema.json

* [many ports] Apply host dependencies

* [yasm-tool] Revert yasm-tool changes to split into PR #16478

* [many ports] Add versions

* [vcpkg.cmake] Revert change applied in other PR

* [boost-modular-build-helper] Merge from master

Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2021-03-26 12:55:34 -07:00
ankurverma85
aa83f4bd2a
[vcpkg] Add x64-windows-static-md to pipeline (#15411)
* Add x64-windows-static-md to pipeline

* Update azure-pipelines.yml

* Update test-modified-ports.ps1

* Replicate skip/fail from x64-windows-static

* Add failures to ci.baseline

* Update ci.baseline.txt

* Update ci.baseline.txt

* Update ci.baseline.txt

* Update ci.baseline.txt

Co-authored-by: ankurv <ankurv@microsoft.com>
2021-01-08 11:55:31 -08:00
Billy O'Neal
229f537c94
[opentracing] Update VMs 2020-12 (#15151)
* Add meson from https://github.com/microsoft/vcpkg/pull/12860/

* Add autoconf-archive from https://github.com/microsoft/vcpkg/pull/13081/

* Add kf5windowsystem libs from https://github.com/microsoft/vcpkg/pull/13467/

* Open the FTP and SFTP ports from https://github.com/microsoft/vcpkg/pull/14412/

* Add libxcb-util0-dev from https://github.com/microsoft/vcpkg/pull/14678/

* Add libasound2-dev from https://github.com/microsoft/vcpkg/pull/14774

* Remove no longer necessary apt-mark calls.

* Update nasm on Linux.

* Fix longstanding bug where Storage was publicly accessible and change to generate SAS token rather than File Share

* Delete no longer necessary azure storage firewall rules.

* Install the newer Windows SDK with the VS installer instead of manually.

* Install the VS2015 and VS2017 compilers.

* Update Powershell-Core to 7.1.0.

* Update source of WDK.

* Update pools.

* [opentracing] Repair arm64-windows failures caused by mojibake in `expected.hpp` and errors in opentracing-cpp's lint for arm64 where it thinks exceptions are disabled when they are enabled.

Fixes:
C:\Dev\vcpkg\buildtrees\opentracing\src\b67575dab0-0250653c81.clean\3rd_party\include\opentracing/expected/expected.hpp(1): warning C4828: The file contains a character starting at offset 0x4a77 that is illegal in the current source character set (codepage 65001).

Fixes:
D:\buildtrees\opentracing\src\b67575dab0-0250653c81.clean\include\opentracing/tracer.h:223:5: error: cannot use 'try' with exceptions disabled [clang-diagnostic-error]
    try {
    ^

* [mmloader] Patch out overrides of CMAKE_C_FLAGS and CMAKE_CXX_FLAGS that inserted /WX.

Note that this port sets /GS-, possibly because it may be for authoring shellcode.

* Bump storage API version to 2020-04-08.
2020-12-22 14:52:58 -08:00
ras0219
5c48bee451
[vcpkg] Add experimental x-azblob binary provider (#13626)
* [vcpkg] Add experimental x-azblob binary provider

* [vcpkg] Test azblob storage provider in CI

* [vcpkg] Address some CR comments from #13639

* [vcpkg] Fixup azure-pipelines

* [vcpkg] Fix regression where the downloaded package is purged before decompressing

* [vcpkg] Further refactor vcpkg::Downloads

* [vcpkg] Enable OSX for x-azblob testing

* [vcpkg] Reduce diff against master

* [vcpkg] Extract Downloads::details::split_uri_view

* [vcpkg] Address PR comments

* [vcpkg] Add testing and metrics for x-azblob

* [vcpkg] Add docs for x-azblob

This includes a note that it is currently experimental

* [vcpkg] Address CR comments

* [vcpkg] Revert pipeline changes except OSX to minimize disruption

Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
Co-authored-by: Billy Robert O'Neal III <bion@microsoft.com>
2020-11-18 12:21:23 -08:00
Jack·Boos·Yu
bf482a9b58
[ampl-mp] Update version, separate port ampl-asl and fix arm build (#14518)
* [ampl-asl] Separate port ampl-mp module asl to ampl-asl

* [ampl-mp] Update version, separate port ampl-asl and fix arm build

* update baseline

* [VMSS] Pre-install ampl-mp:x86-windows

* Update ports/ampl-asl/portfile.cmake
2020-11-12 12:55:11 -08:00
Billy O'Neal
1b1c17de82
[vcpkg baseline] [vs-yasm] Build yasm instead of downloading it to work around memory corruption bugs in yasm itself. (#14003) 2020-10-28 14:26:41 -07:00
Billy O'Neal
4dc9ecd32b
Add MSVC-internal testing scripts to public GitHub. (#13748) 2020-09-28 13:54:10 -07:00
Billy O'Neal
0890b5a25f
[vcpkg] Add build scripts to produce signed vcpkg binaries. (#13508) 2020-09-21 17:27:35 -07:00
Alexander Neumann
5c66f720c0
[cmake] Add cmake port to test qt (#10770)
* [cmake] add port for qt testing

* only add cmcldeps on windows

* [cmake] cleanup and fix portfile for osx build

* [cmake] more cleanup

* move cmake to scripts/test_ports

* add the overlay scripts/test_ports to ci
2020-07-31 16:48:52 -07:00
Billy O'Neal
a28bfe7674
[vcpkg] Remove the tombstones and 'ignore' baseline concepts. (#12197)
This changes our PR builds to treat 'fail' in the ci.baseline.txt as 'skip' instead of using tombstones.

We currently have large numbers of spurious failures that get enshrined in PRs through no fault of a PR author, removing the tombstones concept will fix those by allowing the user to retry. This does mean we accept some risk of not detecting when a port is 'fixed', but that failure is reasonable for us to handle after we see it in CI, but that seems worth it given that it lets us get rid of the tombstone concept.

This also helps out the binary caching feature, because we don't have to figure out how to productize tombstones.
2020-07-02 20:20:07 -07:00
Billy O'Neal
532db1d8f7
[vcpkg] VM Update (#12176)
* [qt5] Add more dependent Linux packages for https://github.com/microsoft/vcpkg/pull/11596/

* [vcpkg] Add LLVM 10 to Windows CI machines for https://github.com/microsoft/vcpkg/pull/11655/

* [vcpkg] Add missing Param block to InstallLLVM function.

* [vcpkg] Add throw after all Write-Error calls for consistency with https://github.com/microsoft/vcpkg/pull/12172
2020-07-01 10:53:10 -07:00
Billy O'Neal
81ae2ed94b
[vcpkg] Add write-only binary caching for CI. (#12130)
Changes to the binary caching spec made as comments over at https://github.com/microsoft/vcpkg/pull/11204#pullrequestreview-438518901
2020-06-26 12:37:53 -07:00
Billy O'Neal
0d37525d75
[vcpkg] Commonize pipelines build powershell and don't use binary caching in CI (rather than PR) builds (#12082) 2020-06-25 16:36:28 -07:00