This reduces bootstrap cost for Windows customers, resolving the issue initially submitted as #12502 .
The `toolsrc` tree was extracted to https://github.com/microsoft/vcpkg-tool. `bootstrap.sh` was changed to download the right source tarball, extract, and build it. This was chosen over the previous attempt, a submodule, over concerns of accidentally destroying people's local modifications.
utilities.cmake: Disable warnings that are firing on Azure Pipelines CI machines due to different Clang version.
expected.h: Fix a bug I introduced in https://github.com/microsoft/vcpkg/pull/15638/ because I was under the impression expected worked like optional (in that value_or_exit for it should be treated as a program bug)
build.cpp: Add (void)s to silence warnings.
binarycaching.cpp: Repair assumption that the current directory is C: which isn't true on the Hosted Azure Pipelines agents.
others: Make unit tests respect %VCPKG_ROOT%, as necessary in the vcpkg_tool repo. Note that this required splitting vcpkgcmdarguments::ImbueFromEnvironment into the once-only process modifying part and the just imbue from environment part.
* [vcpkg] fix checking out git registry ports
* fix the new issues in builtinregistryentry
* fix tests
* fix tests boogaloo
* [vcpkg] Fix issue where baseline is requested for overlay during version resolution
* split BuiltinRegistryEntry into two types
Co-authored-by: Robert Schumacher <ras0219@outlook.com>
* [vcpkg] Miscellaneous internal improvements extracted from #15424
* [vcpkg] CR comments
* [armadillo] Use vcpkg_from_git() to workaround gitlab missing archive
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
Co-authored-by: Billy Robert O'Neal III <bion@microsoft.com>
In build https://dev.azure.com/vcpkg/public/_build/results?buildId=48398
we are getting output like:
```
Error: Building package v-hacd:x64-windows-static failed with: BUILD_FAILED
Elapsed time for package v-hacd:x64-windows-static: 4.156 s
Starting package 1396/1464: v8:x64-windows-static
Building package v8[core]:x64-windows-static...
-- Using msys root at D:/downloads/tools/msys2/969c0913b9df89e1
-- Fetching https://chromium.googlesource.com/v8/v8.git...
CMake Error at scripts/cmake/vcpkg_execute_required_process.cmake:108 (message):
Command failed: C:/agent/externals/git/cmd/git.exe fetch https://chromium.googlesource.com/v8/v8.git 7565e93eb72cea4268028fc20186d415c22b1cff --depth 1 -n
Working Directory: D:/downloads/git-tmp
Error code: 128
See logs for more information:
D:\buildtrees\v8\git-fetch-x64-windows-static-err.log
Call Stack (most recent call first):
scripts/cmake/vcpkg_from_git.cmake:91 (vcpkg_execute_required_process)
ports/v8/portfile.cmake:71 (vcpkg_from_git)
scripts/ports.cmake:128 (include)
Could not copy into file: D:\packages\v8_x64-windows-static\share\v8\vcpkg_abi_info.txt
```
but aren't printing the reason for the copy failure. Added printing of that reason.
* Fix warning on clang version 10.0.0-4ubuntu1
The warning was
```shell
../src/vcpkg/commands.porthistory.cpp:55:14: error: unused function 'is_date' [-Werror,-Wunused-function]
```
* Add environment variable VCPKG_USE_NUGET_CACHE
As the name suggests, this environment variable allow tu use Nuget
cache for Nuget binary caching sources.
* Document NuGet's Cache environment variable
* [vcpkg] Fix regression in error messages with registries/versioning
* [vcpkg] Clean up redundant re-parsing with try_load_overlay_port
* [vcpkg] Deduplicate PathsPortfileProvider
* [vcpkg] Restore uses of Checks::exit_maybe_upgrade after merge
* [vcpkg registries] Add git registries support
* Add git registries support to the registries module of vcpkg.
* add e2e tests for git registries
* fix vcpkg.cmake for registries
* fix CRs, remove a thing
* better error messages
* Billy CRs
* fix Robert's CR comment
* I learned about `-c` today
* format
* fix baseline.json
* failing to find baseline is technically not a bug
* [vcpkg] Change version field in `baseline.json`
* Change name from `version-tag` to `baseline`
* [vcpkg] x-history serializes version scheme
* Update e2e tests
* Update baseline e2e test
* [x-ci-verify-versions] Improve error messages
* Enable CI check for port_versions
* Fix missing and extra line breaks.
* Remove CI check
* [vcpkg] Rewriting CmdLineBuilder (2/n)
I would like, and I think the team would like generally,
to switch to using stuff like `posix_spawn`, as opposed to the
existing use of `system` and `popen`.
This requires a pretty large change to how we use CmdLineBuilder.
The first change we have to make is that the execute functions
_cannot_ take a StringView anymore.
This PR makes that change.
* [vcpkg] Change version field in `baseline.json`
* Change name from `version-tag` to `baseline`
* [vcpkg] x-history serializes version scheme
* Update e2e tests
* Update baseline e2e test
* [vcpkg] Add stuff to the system.process API
* CmdLineBuilder - add rvalue overloads
* Add InWorkingDirectory support
* remove `&&` outside of system.process.cpp
* minor CR
* move the non-InWorkingDirectory functions into the header as inline
* Update toolsrc/include/vcpkg/base/system.process.h
Co-authored-by: Billy O'Neal <bion@microsoft.com>
* fix CmdLineBuilder name
* moar bugfixin
* fix native() location
Co-authored-by: Billy O'Neal <bion@microsoft.com>
* [vcpkg] Add x-ci-verify-versions command
* Code cleanup
* Remove port version splitting from x-history
* Fix wrong message on success
* Parallelize versions file generator
* Use cpu_count()/2 to avoid crashes
* Check db SHA against local SHA
* Check baseline version with x-ci-verify-versions and make baseline generator much faster
* Implement x-add-version to update version db files
* Better checks for x-add-info and make x-ci-verify-versions silent on success
* Use find() instead of [] on maps
* Create version file if does not exist
* Allow redirection of ports/ and port_versions/
* add test ports
* WIP end-to-end tests
* Change pats in e2e tests
* Fix e2e args
* e2e once more
* Pass ersions feature flag to e2e
* Exit with code 1 if there are errors
* Files to test for failure cases
* Update test files
* Add test for x-add-version
* fix redirected ports in last test
* Add CI check (use dummy data)
* Add feature-flags=versions
* Ignore subdirectories inside ports
* Add --verify-git-trees switch
* [vcpkg] Fix build breaks
* [x-ci-verify-versions] PR comments
* [x-add-version] PR comments
* Fix merge conflicts
* Modify tests and pipeline
* Baselines should only have version-string
* Refactor x-add-version
* [vcpkg] Fix help message
* [vcpkg] Fix minor warnings
* `x-add-version --all` doesn't stop on first failure and reduced default verbosity
* [vcpkg] Fix default-baseline
* Load file instead of using paths provider
* Format
* Remove ci test
* Add fish port for testing
* Update version files
* Update fish port to cause SHA discrepancy
* Test for discrepancy between local SHA and declared SHA
* Missing = operator
* Check for error message since x-add-version exits with code 0
* Make x-add-version fail with non-zero exit code if not run with --all
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
Co-authored-by: Robert Schumacher <ras0219@outlook.com>
* Remove port version splitting from x-history
* Parallelize versions file generator
* Use cpu_count()/2 to avoid crashes
* Use generatePortVersionsDb.py output to generate baseline
* Update scripts/generateBaseline.py
Co-authored-by: Adam Johnson <AdamJohnso@gmail.com>
* rename generateBaseline function
* Update toolsrc/src/vcpkg/commands.porthistory.cpp
Co-authored-by: ras0219 <533828+ras0219@users.noreply.github.com>
* Remove unused code
Co-authored-by: Adam Johnson <AdamJohnso@gmail.com>
Co-authored-by: ras0219 <533828+ras0219@users.noreply.github.com>
* [vcpkg] Teach `vcpkg install` `--no-build-missing`
This switch causes failure to restore from cache to abort the build instead of performing a just-in-time build.
This is useful in complex CI systems where the build was expected to have been performed previously, such as via another pipeline.
* [vcpkg] Rename flag to require binary caching
* Add meson from https://github.com/microsoft/vcpkg/pull/12860/
* Add autoconf-archive from https://github.com/microsoft/vcpkg/pull/13081/
* Add kf5windowsystem libs from https://github.com/microsoft/vcpkg/pull/13467/
* Open the FTP and SFTP ports from https://github.com/microsoft/vcpkg/pull/14412/
* Add libxcb-util0-dev from https://github.com/microsoft/vcpkg/pull/14678/
* Add libasound2-dev from https://github.com/microsoft/vcpkg/pull/14774
* Remove no longer necessary apt-mark calls.
* Update nasm on Linux.
* Fix longstanding bug where Storage was publicly accessible and change to generate SAS token rather than File Share
* Delete no longer necessary azure storage firewall rules.
* Install the newer Windows SDK with the VS installer instead of manually.
* Install the VS2015 and VS2017 compilers.
* Update Powershell-Core to 7.1.0.
* Update source of WDK.
* Update pools.
* [opentracing] Repair arm64-windows failures caused by mojibake in `expected.hpp` and errors in opentracing-cpp's lint for arm64 where it thinks exceptions are disabled when they are enabled.
Fixes:
C:\Dev\vcpkg\buildtrees\opentracing\src\b67575dab0-0250653c81.clean\3rd_party\include\opentracing/expected/expected.hpp(1): warning C4828: The file contains a character starting at offset 0x4a77 that is illegal in the current source character set (codepage 65001).
Fixes:
D:\buildtrees\opentracing\src\b67575dab0-0250653c81.clean\include\opentracing/tracer.h:223:5: error: cannot use 'try' with exceptions disabled [clang-diagnostic-error]
try {
^
* [mmloader] Patch out overrides of CMAKE_C_FLAGS and CMAKE_CXX_FLAGS that inserted /WX.
Note that this port sets /GS-, possibly because it may be for authoring shellcode.
* Bump storage API version to 2020-04-08.
* [vcpkg registries] support versions
This PR merges the Registries changes and the versioning changes, so that one can use both at the same time.
There is one major difference between this PR and the RFC (#13590), which is that instead of version files looking like:
```json
[
...
]
```
version files look like:
```
{
"versions": [
...
]
}
```
this is to support interop between this PR and existing demos and the like;
fixing this, along with perhaps renaming `port_versions` to `port-versions` should be done after this is merged,
should be a trivial change.
Other changes:
* Changed Strings::replace_all to use more StringView
* Introduced Strings::inplace_replace_all for the common x = replace_all(move(x) pattern and tests
-> Also fixed bug if the search string was empty consuming infinite memory!
* Added many missing {}s in binarycaching.cpp
* [vcpkg] Remove extra indirection in IVersionedPortfileProvider/IBaselineProvider
* [vcpkg] Address CR comment
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
* [vcpkg] Refactor end-to-end tests
* [vcpkg] Cherry-pick x-builtin-ports-root from #14999
* [vcpkg] Move create test from unit tests to e2e
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
