147 Commits

Author SHA1 Message Date
Billy O'Neal
b77f20ec4c
Recombine osx and normal ci. (#23144)
* Some of the Windows runs have started to exceed 24 hours now as well as the MacOS ones as we have added more ports.
* We are no longer trying to constantly "psychic debug" why ports are failing; CI passes somewhat regularly now, and the failures we get seem to be much more deterministic.
* We're still budget constrained on Azure resources (and have been a bit over budget)

To that end, we're going back off nightly CI runs to Monday Wednesday Friday (like osx already is), so we no longer need separate bits for them.
2022-02-16 19:55:46 -08:00
Billy O'Neal
c93faafe6d
Update vcpkg-tool to 2022-02-11 (#23059)
* Update vcpkg-tool to 2022-02-11

* Fix agent pool.
2022-02-13 21:57:51 -08:00
Billy O'Neal
898d6b1a1c
Minor bugfixes to MacOS deployment readme. (#23062)
* Minor bugfixes to MacOS deployment readme.

* Update vsts-agent to 2.198.3
* Remove rimraf of ~/vagrant/vcpkg-eg-mac
* Add fetch and branch switch before running Install-Prerequisites.ps1 to make sure it's updated
* Clarify "the fileshare" as ~/vagrant/share
* Clarify that the date `Setup-VagrantMachines.ps1` wants is the box version date.
* `Setup-VagrantMachines.ps1` parameter to be DevopsPat rather than Pat

* nicole updates

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
2022-02-13 21:57:22 -08:00
nicole mazzuca
47c5701747
[macos ci] 2022-02-04 update (#22942)
* add moar ram (no PR)

* Provision autoconf-archive on osx (PR #22872)

* [mpfr] Support Linux and OSX (PR #22845)

* VM software versions

* fix minor schema bug

* update pool

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
Co-authored-by: Kai Pastor <dg0yt@darc.de>
Co-authored-by: JackBoosY <yuzaiyang@beyondsoft.com>
2022-02-09 13:25:57 -08:00
Billy O'Neal
39b17cf84c
Update VMs for Patch Tuesday and fix pwsh CVE. (#22494) 2022-01-13 16:12:55 -08:00
nicole mazzuca
5edd518dc3
[macos ci] update to 2022-01-03 (#22314)
* [macos ci] update to 2022-01-03

* update the instructions

* forgot to update azure-pipelines-osx.yml

* install parallels in Install-Prerequisites.ps1

* oops

* fix ace, libsoundio, and libpqxx

* tensorflow doesn't yet support macOS 12
2022-01-10 16:08:51 -08:00
Billy O'Neal
c6ffd8eeb2
[vcpkg] Update VMs for December 2021 Patch Tuesday (#22013)
* Add Microsoft.VisualStudio.ComponentGroup.UWP.VC.BuildTools to VS as requested by https://github.com/microsoft/vcpkg/issues/19554

* Cherry pick python changes from https://github.com/microsoft/vcpkg/pull/21912

* Update linux pool.

* Update windows pool.
2021-12-14 16:15:33 -08:00
Joachim Gehweiler
a9ff505c5b
[vcpkg baseline][tensorflow] setup python venv and update Linux VMs (#21912)
* Revert "incorporate changes from microsoft:master"

* Revert "Revert "incorporate changes from microsoft:master""

* preliminary approach to solve pip issue

* fix python venv

* fix macOS build command again

* bump version

* x-add-version

* Fix spelling of prerequisites.

* Add python3-venv

* Update linux pool.

Co-authored-by: Joachim Gehweiler <joachim@Joachims-iMac.local>
Co-authored-by: Billy Robert O'Neal III <bion@microsoft.com>
2021-12-13 17:46:40 -08:00
Billy O'Neal
d70768e374
[vcpkg] Change osx to run every 48 hours rather than every 24. (#21954)
* Change osx to run every 48 hours rather than every 24.

* Back out more former-tensorflow-separation yaml.
2021-12-13 12:54:16 -08:00
Dawid Wróbel
a2fcb03749
[libmount] New port (#21491)
* [libmount] new port

* [libmount] update versions

* [kf5solid] add libmount feature

* [kf5solid] update versions

* [kf5kio] add libmount dependency

* [kf5kio] update versions

* [libmount] Remove libmount-dev from Linux CI image

* [glib] Add libmount feature

* [glib] update versions

* [ECM] fix linking against static LibMount

* [ECM] update versions

* [kf5solid] make libmount feature a default on Linux

* [kf5solid, kf5kio] update versions
2021-11-23 20:54:53 -08:00
Jack·Boos·Yu
68a2521617
[vcpkg baseline][rtaudio] Add feature alsa and fix dependency (#21613)
* [vcpkg baseline][rtaudio] Add feature alsa and fix dependency

* version

* Remove not needed apt item
2021-11-22 20:33:13 -08:00
NN
34b2b8603e
Update to PowerShell 7.2.0 (#21339) 2021-11-22 14:29:40 -08:00
Billy O'Neal
e46b12b825
Update VMs for November Patch Tuesday (#21309)
* Apply VM updates for:

* https://github.com/microsoft/vcpkg/pull/13994
* https://github.com/microsoft/vcpkg/pull/20654
* https://github.com/microsoft/vcpkg/pull/20938

* Be more defensive in AzVmRunCommandWithRetries.

* Update Pools.
2021-11-10 14:07:00 -08:00
Kai Pastor
eea02380aa
[ci] Remove unaffected ports from pull request builds (#21078)
* Rewrite to single invovation of 'vcpkg ci'

* Split caching args from common args

* Use parent hashes for pull request CI builds

* Completely switch to parent revision state and back
2021-11-01 12:18:28 -07:00
Billy O'Neal
4474aba1e7
[ci] Attempt to make CI more reliable by using robocopy to delete (#20889) 2021-10-20 22:51:29 -07:00
Billy O'Neal
078f3e51ef
[ci] Skip attempting to upload failure logs if there is no failure logs directory. (#20826) 2021-10-18 15:04:49 -07:00
Billy O'Neal
699c8779f1
[vcpkg] Update VMs for October Patch Tuesday and Server 2022 (#20730)
* Patch tuesday and switch to Server 2022.

* Update pools.
2021-10-16 12:14:53 -07:00
Billy O'Neal
f670aa5657
Use the correct SAS token for macos and fix tensorflow build failing on success. (#20512)
* Use the correct SAS token for macos and fix tensorflow build failing on success.

* Fix binary cache.
2021-10-04 11:52:47 -07:00
nicole mazzuca
4187499295
[macos ci] oops that was a dumb bug (#20448) 2021-09-30 19:52:23 -07:00
nicole mazzuca
45315bc3aa
[macos ci] move out tensorflow (#20388)
* [macos ci] move out tensorflow

* rename ExtraPorts and CorePorts

Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
Co-authored-by: Billy Robert O'Neal III <bion@microsoft.com>
2021-09-29 13:15:25 -07:00
Kai Pastor
827a54c404
[ci] Don't emit warnings for absence of failure logs (#20316)
* No warning for absence of failure logs

* Update condition for osx and linux

Co-authored-by: Billy Robert O'Neal III <bion@microsoft.com>
2021-09-28 14:00:33 -07:00
Billy O'Neal
09980780d8
[ci] Move formatting and doc checks into x86-windows (#20402)
* [ci] Move formatting and doc checks into x86-windows to avoid burning a whole job for that 1 run, don't run file lists generation in PRs.

* Deduplicate x86-windows and restore PR file lists, as requested/suggested by @dg0yt

* More quotes

Co-authored-by: nicole mazzuca <83086508+strega-nil-ms@users.noreply.github.com>

Co-authored-by: nicole mazzuca <83086508+strega-nil-ms@users.noreply.github.com>
2021-09-28 13:51:42 -07:00
Billy O'Neal
93a8320399
Back out "cascades" enforcement because it caused too many stealth merge conflicts. (#20181)
Mostly reverts https://github.com/microsoft/vcpkg/pull/20140

See https://github.com/microsoft/vcpkg/discussions/20180 for further discussion.
2021-09-15 11:39:39 -07:00
Billy O'Neal
5395ad9a4f
Add "cascades" enforcement (#20140)
* Add "cascades" enforcement as requested by https://github.com/microsoft/vcpkg/pull/20074#issuecomment-916565430

* Add counts.

* Fix arm-uwp count.

* Update counts.

* Update counts
2021-09-14 17:46:38 -07:00
Billy O'Neal
c26477297c
Update pwsh to 7.1.14, separate image minting from vmss minting, and update VMs. (#20064)
* Update Powershell-Core to 7.1.4.

* Add deployment of .NET 4.7.2 targeting pack, extracted from https://github.com/microsoft/vcpkg/pull/19320

* Separate image minting from vmss minting.

* Update pools.

* Fix image minting typo on Windows.
2021-09-11 13:43:19 -07:00
Billy O'Neal
f78c96f706
Decouple the storage account used for binary caching from the images. (#19912)
* Decouple the storage account used for binary caching from the images.

We are being forced to change infrastructure somewhat more frequently than we would like, so change the binary caches to use a shared storage account like we already use for asset caching.

* blob blob blob

* Add quotes

* Avoid name stomping.
2021-09-03 00:02:29 -07:00
Billy O'Neal
e22f9d7d70
Update VMs for Patch Tuesday, 2021-08-11 (#19508)
* Deploy SDKs with full features first to ensure we get Debugging Tools for Windows.

* Update VMs for Patch Tuesday, 2021-08-11
2021-08-13 16:12:24 -07:00
nicole mazzuca
ff190a561a
[macos ci] update 2021-07-27 (#19207)
* update 2021-07-27

macOS -> 11.5.1, XCode -> 12.5.1

* update azure-pipelines

* update sha of macfuse

* change how macos-ci-base works

* fix build errors

* fix itpp:linux

* more fixes

* remove tab

* allow version changes in all the remove/rename-version patches

* fix libunistring for real

* robert CR
2021-08-04 13:27:40 -07:00
Billy O'Neal
6b6b1e3b0b
Avoid rmdir the downloads directory is nonexistent. (#19314) 2021-08-02 15:16:08 -07:00
Billy O'Neal
403c952a0c
Clean the downloads directory every CI run. (#19311)
Now that vcpkg has artifact caching, meaning downloading artifacts usually doesn't have to leave the datacenter, there's no need to potentially dirty different runs with leftover downloads from each other.

This resulted in some "impossible" results in https://github.com/microsoft/vcpkg/pull/18687 where a previous PR run accidentially downloaded the Windows version of nodejs to /mnt/vcpkg-ci/downloads/node-v14.17.4-linux-x64.tar.xz
2021-08-02 14:15:48 -07:00
Kai Pastor
e5fdfd3d36
Add AZP logging markup for versioning errors (#19104) 2021-07-28 12:15:10 -07:00
Billy O'Neal
b67fab9861
Do not emit 'PASSING, REMOVE FROM FAIL LIST' results when using vcpkg to test prerelease versions of MSVC++ (#19011) 2021-07-19 19:55:00 -07:00
Billy O'Neal
f698b0e8be
Change VM size to Standard_D32ds_v4. For some reason they are cheaper than D16 right now??? (#18953)
Remove no longer used "unstable" scaffolding.
Use ephemeral OS disks for better latency.
Add ability to change whether temp disks are used or explicitly provisioned disks are used; explicitly provisioned disks turned out to be more expensive than expected before.
Disable Windows Updates in the scale set because the machines are very short lived; working on a mechanism of patching the images instead.
2021-07-15 13:47:58 -07:00
Billy O'Neal
957d2024b8
Update to Ubuntu 20.04. (#18892)
* Update Ubuntu to 20.04.

* [tfhe] Disable -Wall -Werror

* [sophus] Disable -Werror.

* Fix typo in deployment of CUDA.

* Add libxxf86vm-dev for opensubdiv.

* [opensubdiv] Fix OpenCL search on Ubuntu 20.04, and document libxxf86vm-dev dependency

* [msgpack11] Disable -Werror.

* [qtwayland, qt5wayland] Add system library notice.

* Add python-is-python3 to unbreak tensorflow, v8, and others

* [graphqlparser] Remove directive rejected by Bison 3.5.1

* [ogdf] Mark as failing in ci.baseline.txt. Issue filed https://github.com/microsoft/vcpkg/issues/18936

* [bde] Mark failing in ci.baseline.txt, issue filed https://github.com/microsoft/vcpkg/issues/18937

* [soem] Patch out -Werror

* Update pool.
2021-07-14 12:46:05 -07:00
Billy O'Neal
7d169b467c
Update pool to get print spooler CVE fix. (#18891) 2021-07-10 16:47:55 -07:00
Billy Robert ONeal III
d2717c38db [vcpkg internal] Update compiler test scripts to use asset caching. 2021-06-30 00:51:32 -07:00
Billy O'Neal
75beef03d6
Update vcpkg tool to resolve leak of the asset cache token (#18720) 2021-06-29 13:47:39 -07:00
Billy Robert O'Neal III
b504547c13 [vcpkg-unstable] Update agent pool name 2021-06-24 09:53:27 -07:00
Billy O'Neal
2ed44b4546
Update VMs and pick up VS2019 16.10 (#18233)
* Cherry-pick https://github.com/microsoft/vcpkg/pull/15598

* Hook deploy-inteloneapi into create-vmss.ps1.

* Add script to resolve https://github.com/microsoft/vcpkg/issues/17521

* Move tls settings deployment to the front and respond to script triggering a reboot.

* Go back to provisioning an extra disk to workaround https://github.com/microsoft/vcpkg/issues/18379

* Disallow public access to blob storage and require TLS 1.2.

* Update Pools.

* Update tool to 2021-06-19

* [simage] Skip simage on uwp platforms as it appears broken by 16.10.

* [tensorflow-cc] Skip because changes in our MacOS hardware broke the port.
2021-06-22 19:16:04 -07:00
ras0219
8c497bb647
[vcpkg] Add experimental $X_VCPKG_ASSET_SOURCES for source caching (#13639)
* [vcpkg] Add experimental $VCPKG_X_READWRITE_MIRROR_URL_TEMPLATE for source caching

* [vcpkg] Use prerelease version for testing

* [docs] Add assetcaching.md

* Revert bootstrap changes

* [vcpkg_download_distfile] Deprecate SILENT_EXIT

* cmake_parse_arguments() always defines option arguments

* Adjust CI baseline

* [vcpkg_download_distfile] Address CR comments, restore ARIA2

* [CI] Add source caching to Linux and OSX CI passes

* [ci.baseline.txt] Skip chartdir on more platforms
2021-06-04 16:48:26 -07:00
Phoebe
117cf538f4
[vcpkg] Remove workaround in Ci pipeline (#18119) 2021-05-26 13:55:22 -07:00
nicole mazzuca
0e914ba681
[scripts-audit] Rollup 2021-05-21 (#18112)
* [scripts-audit] vcpkg_acquire_msys (#17687)

* [vcpkg.cmake] Change find_package function -> macro in order to shorten trace (#17997)

switch from function to macro

* [scripts-audit] z_vcpkg_apply_patches (#18013)

* [scripts-audit] z_vcpkg_escape_regex_control_characters (#18013)

* [scripts-audit] vcpkg_fail_port_install (#18054)

Note that this change allows existing failures;
it changes MATCHES to STREQUAL, meaning that
vcpkg_fail_port_install(ON_ARCH "arm") now succeeds on arm64;
this change in behavior is fine (IMO) since it strictly succeeds
when it used to fail.
2021-05-26 13:40:48 -07:00
Robert Schumacher
4e83da0887
[vcpkg] Update vcpkg tool version 2021-05-05 (#17689)
* [vcpkg] Update vcpkg tool version 2021-05-05

* [vcpkg] Hotfix bug in Microsoft/vcpkg-tool#34

* [vcpkg-ci] Ignore results from non-target triplets

* [protobuf] Fix regression introduced in #16904

* [vcpkg] Fix bug where 'staticcrt' was ignored

Co-authored-by: Robert Schumacher <ras0219@outlook.com>
2021-05-06 07:12:43 -07:00
nicole mazzuca
e6cabdece5
[ci] Update macOS to 11 (#17376)
* start 2021-04-16 process

Major new things:
* update to macOS Big Sur (11.*)
* switch from VirtualBox to Parallels due to ^ (and also ARM)

Minor new things:
* update from xcode CLT 12 to 12.4

This PR includes new stuff for creating the base box for parallels.
Still to do: using the new box type.

* update the vagrantfile stuff

* link the CI to the new version

* modify how macOS boxes are made

the Vagrantfile for the final VM will only download the azure pipeline stuff

this allows us to keep the versions of brew applications stable

* fix cpus and memory

* add vagrant plugins to installables

* Skip cppgraphqlgen ICE.

* [sdformat6] Remove unneeded include(FindBoost) which causes problems when the system cmake version doesn't match the one deployed by vcpkg.

* switch to dmg for installing osxfuse/sshfs

* Set cores to 11 (leaving 1 thread for host)

Co-authored-by: Billy Robert ONeal III <bion@microsoft.com>
2021-04-29 07:39:04 -07:00
Billy O'Neal
a9b27ed5df
Update VMs, CMake to 3.20.1, CUDA to 11.3, and pwsh to 7.1.3 (#17331)
* Update to CUDA 11.3 on Windows.

* Update PowerShell to 7.1.3.

* Update CUDA to 11.3 on Linux.

* "Explode" VM provisioning scripts for more consistent feedback during deploy. This resolves the deployment script often hanging with no feedback as to why.

* [libdatachannel] Fix use of deprecated result_type typedef.

Submitted upstream as https://github.com/paullouisageneau/libdatachannel/pull/413

* [libvpx] Get the libvpx outputs from the correct place.

(Perhaps VS2019 version 16.10 moved where these are placed? I've been defensive and left an attempt to find from the old location in place.)

* [chromaprint] Support implementations where lrintf is an intrinsic.

* Add provision-entire-image script.

* [cudnn] Disable download-on-the-fly due to licensing concerns.

* Add libnccl to Linux VMs.

* [wangle] Disable x64-windows due to an ICE.

* [cmake] Update cmake to 3.20.1 to avoid https://gitlab.kitware.com/cmake/cmake/-/issues/21571 race

* [libudis86] Fix passing a bogus working directory which fails on CMake 3.20.x

* [dartsim] Disable unit tests, examples, and tutorials, some of which have CMake authoring errors rejected by 3.20.1.

* Add thrust to the cuda installees.

* [tensorflow] Put .bzl in CURRENT_BUILDTREES_DIR to avoid running out of disk space in CI and to respect --clean-after-build.

* [dimcli] Skip port broken by changes in VS2019 project system.

* [upb] Disable an additional warning.

* [libhv] Fix typo DISABLE_PARALLEL => DISABLE_PARALLEL_CONFIGURE

* Update pools
2021-04-26 10:27:45 -07:00
ras0219
b5bb1511f0
[many ports] Apply host dependencies (#16479)
* [vcpkg] Add VCPKG_HOST_TRIPLET. Improve vcpkg.schema.json

* [many ports] Apply host dependencies

* [yasm-tool] Revert yasm-tool changes to split into PR #16478

* [many ports] Add versions

* [vcpkg.cmake] Revert change applied in other PR

* [boost-modular-build-helper] Merge from master

Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2021-03-26 12:55:34 -07:00
Phoebe
02daf011f0
[Azure pipeline] Upload file list logs for unstable testing (#16492)
* [Azure pipeline] Upload file list logs for unstable testing

* Fix indentation
2021-03-10 14:05:57 -06:00
Billy O'Neal
aa60b7efa5
[vcpkg] Download vcpkg.exe rather than building it in bootstrap on Windows. (#15474)
This reduces bootstrap cost for Windows customers, resolving the issue initially submitted as #12502 .

The `toolsrc` tree was extracted to https://github.com/microsoft/vcpkg-tool. `bootstrap.sh` was changed to download the right source tarball, extract, and build it. This was chosen over the previous attempt, a submodule, over concerns of accidentally destroying people's local modifications.
2021-02-04 10:15:44 -08:00
nicole mazzuca
dc4d1b735a
[vcpkg] fix checking out git registry ports (#16009)
* [vcpkg] fix checking out git registry ports

* fix the new issues in builtinregistryentry

* fix tests

* fix tests boogaloo

* [vcpkg] Fix issue where baseline is requested for overlay during version resolution

* split BuiltinRegistryEntry into two types

Co-authored-by: Robert Schumacher <ras0219@outlook.com>
2021-02-03 10:03:44 -08:00
ras0219
5793c4bd9f
[vcpkg] Miscellaneous internal improvements extracted from #15424 (#15677)
* [vcpkg] Miscellaneous internal improvements extracted from #15424

* [vcpkg] CR comments

* [armadillo] Use vcpkg_from_git() to workaround gitlab missing archive

Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
Co-authored-by: Billy Robert O'Neal III <bion@microsoft.com>
2021-02-02 15:00:33 -08:00