* [vcpkg registries] support versions
This PR merges the Registries changes and the versioning changes, so that one can use both at the same time.
There is one major difference between this PR and the RFC (#13590), which is that instead of version files looking like:
```json
[
...
]
```
version files look like:
```
{
"versions": [
...
]
}
```
this is to support interop between this PR and existing demos and the like;
fixing this, along with perhaps renaming `port_versions` to `port-versions` should be done after this is merged,
should be a trivial change.
Other changes:
* Changed Strings::replace_all to use more StringView
* Introduced Strings::inplace_replace_all for the common x = replace_all(move(x) pattern and tests
-> Also fixed bug if the search string was empty consuming infinite memory!
* Added many missing {}s in binarycaching.cpp
* [vcpkg] Remove extra indirection in IVersionedPortfileProvider/IBaselineProvider
* [vcpkg] Address CR comment
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
* [vcpkg] Refactor end-to-end tests
* [vcpkg] Cherry-pick x-builtin-ports-root from #14999
* [vcpkg] Move create test from unit tests to e2e
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
* [vcpkg] Improve error messages for incomplete Mono
* [vcpkg] Add mono upgrade notice for Ubuntu 18.04 users
* [vcpkg] Fix always failing to check CMake version due to preferring 'error' constructor in ExpectedS
* [vcpkg] Eagerly display the 'Ubuntu needs new mono' message to avoid users needing to install it twice
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
* [vcpkg] Add semver versioning scheme
* Remove unnecessary code
* Fix SemVer comparison and add sorting test
* Add date scheme
* PR comments
* Use a different column for date and semver schemes.
* Use locale agnostic conversion to long
* Add tests for version scheme change
* Validate version strings before parsing
* Format
* Improve error messages
* PR comments
* PR comments pt. 2
* [vcpkg] Refactor deserializers to reduce duplicate functionality
* [vcpkg] Error on '#' in version strings to avoid confusion with port-version
* [vcpkg] Improve error message
* [vcpkg] Reorder field output
* [vcpkg] Fix tests
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
* [vcpkg] Output versions during install plans
* [vcpkg] Display versions for already-installed packages
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
* Check for the 'versions' feature flag while installing a manifest
* Implement two-level fallback for Baselines:
* First, fall back to current repo's baseline.json
* Second, fall back to the ports in ports/
* Optimize checking out baseline.json via `git show`
* Detect presence of '$x-default-baseline' in the manifest and use that as the baseline commit (to be replaced by Federation functionality)
* If port_versions/x-/xyz.json is not found, fall back to ports in ports/
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
* [vcpkg] Add option cmake-args to pass additional args to the script execution
* apply clang format to install.cpp
* rename command to x-cmake-args
* imply editable if x-cmake-args are not empty.
* don't use bitwise or
* add a definition for CMAKE_SCRIPT_ARG
Co-authored-by: wangli28 <wangli28@beyondsoft.com>
* [vcpkg] Add experimental x-azblob binary provider
* [vcpkg] Test azblob storage provider in CI
* [vcpkg] Address some CR comments from #13639
* [vcpkg] Fixup azure-pipelines
* [vcpkg] Fix regression where the downloaded package is purged before decompressing
* [vcpkg] Further refactor vcpkg::Downloads
* [vcpkg] Enable OSX for x-azblob testing
* [vcpkg] Reduce diff against master
* [vcpkg] Extract Downloads::details::split_uri_view
* [vcpkg] Address PR comments
* [vcpkg] Add testing and metrics for x-azblob
* [vcpkg] Add docs for x-azblob
This includes a note that it is currently experimental
* [vcpkg] Address CR comments
* [vcpkg] Revert pipeline changes except OSX to minimize disruption
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
Co-authored-by: Billy Robert O'Neal III <bion@microsoft.com>
* [llvm] install tools in tools/llvm
* remove the giant commented out block of code
* update port-version
* hopefully fix the port?
There are still some issues, namely that the python scripts and DLLs that are copied into `tools/llvm`
are not considered to be "installed by llvm", and thus are not removed when llvm is removed
* format!
* apparently REGEX REPLACE fails if a thing doesn't match
* fix LLVM_REMOVE_EXTENSION_REGEX on windows
* actually read the cmake regex docs...
* fix the name of the variable
* turns out CMAKE_MATCH_1 is the one I want
* need to update VERSION for new policy
* stop removing debug/bin
* fix faulty merge
* [vcpkg] Add disable-lock options
This should fix#14281 at least as a stopgap.
Additionally, adds better errors.
* billy CRs
* change from "disable-lock" to "allow-spurious-lock-failures"
* billy cr
* fix build issues occurring from default bazel config being used because customized bazel config is stored in wrong directory
* [tensorflow-cc] Update CONTROL and ci.baseline.txt
* fix also applies to windows static build
* fix debug and static builds as well as library naming for non-linux
- add patch to fix debug builds
- add patch to fix exports for static linking
- really build debug (instead of cloning release)
- override bazel build options for debug (work around bazel bug)
- bazel doesn't support static libraries: work around by building dynamic library and constructing static linkage commands from build log
- Windows .pdb file can't be >4GB even on x64: work around using reduced debug information
- Windows doesn't support .lib files >4GB even on x64, so split into multiple libs
- vcpkg requires equal amount of libs for debug and release: work around using handcrafted empty dummy libs
- fix naming of libs (.dll on Windows and .dylib on macOS)
* upgrade tensorflow from v1.14 to v2.3
- adapt patch files to tensorflow code changes
- update bazel from v0.25.2 to v3.1
- on Windows use python installed on the host instead of embedded python obtained via vcpkg because embedded python lacks pip, which we need to obtain numpy
- on Windows add MSYS2 to the PATH so that bazel tools can access MSYS2 GIT
- add support for custom CA certificates when using HTTPS_PROXY
* fix execute process macro
The existing implementation totally screwed up commands if the command's arguments contained semicolons (this is the case, e.g., in the FindPython modules of the cmake distribution).
* extend overriden execute_process to more than one COMMAND as there actually are use cases for this
* added another patch required for tensorflow v2.3, fixed path and working directory
* Revert "incorporate changes from microsoft:master"
* Revert "Revert "incorporate changes from microsoft:master""
* final fixes for static build + improving out messages
* enabling linux and osx in CI to see if it works now
* simplified code, fixed version numbers, fixed generated include cmake file
* fix failing postbuild check on handcrafted empty dummy library by spreading the last real libraries contents over the required number of libraries
* remove dead code commit by mistake again
* improvements from code review
* cleaner fix for debug code
* find pip3 in PATH (PYTHON3_DIR apparently not valid for pip3)
* fix error in python helper script
* fix wrong libname in postbuild script
* fix python detection + switch to python on msys2 (instead of embedded python) for Windows as we need numpy
* fix order of arguments
* fix command (it may contain spaces such as C:\Program Files\...)
* revert last commit (root cause for CI failures is something different: there are line breaks in path)
* fix regex comparision
(value needs to be escaped as it may contains regex special characters such as brackets, eg C:/Program Files (x86)/...)
* fix linebreaks in generated file
* fix CRT linkage
(macOS doesn't support static CRT linkage; it's set to dynamic even static target triplets for macOS and linux)
* refactor implemenation to avoid as much code duplication as possible -- algorithmically identical
* fix version numbers in helper scripts
* enable work-around for Windows until bazel fix is available
* install missing python3-pip on linux
* fix linux build by patching
* apply timeout feature now available via merged master branch
* correct linux build patch
* improve debug build patches
(no functional difference because LOG(FATAL, ...) macro internally anyway calls abort(), which the compiler doesn't detect in debug mode...
* improve linux patch
* temporarily add debug to inspect what's going on on macOS CI
* remove temporary debug code and fix static linking scripts for linux and macOS
* fix regex escaping
* fix ambiguous match while grepping for the framework link command
* extend fix of ambiguous match while grepping for the framework link command
* fix what merge of master broke
* fix more what got broken by merging master
(all packages and their dependencies are now maintained manually instead of using pacman...)
* remove "unofficial" from filename
* added switch do distinct classic and manifest mode when generating config.cmake file
* create symlinks for libraries without version number
* fix linux postbuild script
* temporarily disable code making problems
* add note for linking on Linux and macOS
* forget to add README file in previous commit
* add file forgotton in macro fixing patch
* fix python library path
* fix macOS static link command
* update linkage instructions in README
* Update ports/tensorflow-cc/CONTROL
* Update ports/tensorflow-cc/portfile.cmake
* Update scripts/ci.baseline.txt
* use vcpkg_execute_required_process
* pass C_FLAGS and CXX_FLAGS to bazel
* fix INTERFACE_INCLUDE_DIRECTORIES
* fix optional c/cxx arguments
* also add linker opts
* update README
* merge static libs into one
to support force_load (cannot force_load both due to duplicate symbols)
* update README
* quote python path (it might contain spaces that don't get escaped inside outer quotes of bash command)
* fix python path also for static build
* add arm(64) as currently unsupported arch
* bazel 3.7 is available -> remove workaround
* update README, remove necessary c-ares from deps
* update msys package
* add uwp specific options, and minor general improvements
* fix string replace
* fix control file and windows path separator
* revert backslashes-fix -- the root cause was missing .exe extension
* upgrade to tf 2.3.1
* fix hard-coded version
* remove uwp work-in-progress code so that PR can be merged
* update README and print out usage info in portfile
* Update ports/tensorflow-cc/README-linux
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/README-linux
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/README-linux
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/README-linux
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/README-linux
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/README-windows
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/portfile.cmake
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/portfile.cmake
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/portfile.cmake
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/portfile.cmake
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/portfile.cmake
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* improve usage hints as discussed in review
* add comment
* apply changes from review
* make additional compiler / linker args space-proof
* Update ports/tensorflow-cc/README-macos
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/tensorflow-cc-config-shared.cmake.in
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/tensorflow-cc-config-shared.cmake.in
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/README-linux
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* Update ports/tensorflow-cc/README-macos
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* apply changes from code review
* maybe fix the config files
* rob.maynard CRs
* fix windows static lib naming for first part
* Update ports/tensorflow-cc/generate_static_link_cmd_windows.py
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* apply rob maynards 1st suggestion
* nicole CRs
* format/fix-compile
* fix missing string termination
* prefer IMPORTED_LOCATION over IMPORTED_LOCATION_RELEASE to have default fall-back
* hopefully fix the issue where no libraries are generated
* final stuff
Co-authored-by: Gehweiler <Joachim_Gehweiler@McAfee.com>
Co-authored-by: wangli28 <wangli28@beyondsoft.com>
Co-authored-by: Jack·Boos·Yu <47264268+JackBoosY@users.noreply.github.com>
Co-authored-by: Joachim Gehweiler <joachim@Joachims-iMac.local>
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
* [vcpkg wip] start implementation of registries pt. 2
* Remove the `ports` field of `VcpkgPaths`
This is an implementation detail, and so we want to make sure that
it's obvious that this is an internal detail; thus, we add a new function
`builtin_ports_directory()`,
which returns the directory where the builtin ports backing store is.
* continue WIP
* wip
* It works!
* format
* fix some issues
* switch from function static to DelayedInit
* fix lexically_normal for experimental::filesystem
* format
* fix missing include
* add STL notice
* whee error handling
* moar error handling!
* ignore extra files in registries
* formatting
* Billy CRs
* Update toolsrc/include/vcpkg/versiont.h
* fix add_filename test
* fix tests
* remove unused function add_filename
* [vcpkg] Add script to generate ports versions history
* [vcpkg] Fix formatting
* Fetch port versions from commit ID
* Use global --x-json switch
* Use --no-checkout when cloning secondary instance
* Clone from local repository instead of from GitHub
* Use CmdLineBuilder to build git commands
* Use CmdLineBuilder and reduce repeated code
* Fetch version at baseline and code cleanup
* Guess version scheme from old CONTROL files
* Rename version db generator script
* Simplify x-history json output
* Use CONTROL/manifest parsers on x-history
* Use git-tree instaed of commit-id
* Remove 'ports' field from root object
* Clean up code
* More code cleanup
* Improve port version detection
* Improve generator logging
* Do not ignore parsing errors in CONTROL files
* PR review comments in Python script
* Fix subprocess.run() calls
* Make `canonicalize()` return error instead of terminating
* [vcpkg] Add tests for new test_parse_control_file paths
* Remove unnecessary std::move() calls
* Fix formatting
* Python formatting
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
* [vcpkg] Add `versions` feature flag and version field manifest parsing
* Introduce FeatureFlagSettings struct to more easily access feature flags throughout the program
* To avoid users accidentally starting to write "version" instead of "version-string" in their manifests, vcpkg explicitly detects and prevents usage of ports with schemes other than "String"
* Drive-by fix of copiable SourceControlFileLocation and an exposed use-after-move bug
This code is largely extracted from PR #13777
Co-authored-by: Victor Romero <romerosanchezv@gmail.com>
* [vcpkg] Address CR Comments. Fix test crash on Linux.
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
Co-authored-by: Victor Romero <romerosanchezv@gmail.com>
* [vcpkg] Split vcpkg/base/json.h into vcpkg/base/jsonreader.h
* [vcpkg] Extract definitions of Configuration-Deserializer (& friends)
These types are only used by VcpkgPaths during the initial parse.
* [vcpkg] Introduce levenshtein-distance suggestions for json errors
* [vcpkg] Fix regression in supports handling
* [vcpkg] Fix signed/unsigned mismatch
* [vcpkg] Address CR comments
* [vcpkg] Address CR comments
* Fix compiler error from merge conflict.
* [vcpkg] Change parameters of Reader::check_for_unexpected_fields to better match declaration
* [vcpkg] Improve errors from features set
* [vcpkg] Fix includes
* [vcpkg] Reuse code
* [vcpkg] Check the "name" field always to maximize error information
* [docs] Improve english phrasing in manifests.md
* [vcpkg] Correct docs link for manifests
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
Co-authored-by: Billy Robert O'Neal III <bion@microsoft.com>
* [vcpkg] Avoid computing triplet ABIs for editable packages
* [vcpkg] Address PR comments
* [vcpkg-end-to-end-tests] Add tests for --no-binarycaching, binarycaching by default, and --editable
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
* [vcpkg] Enable recursive vcpkg calls
Via envvars VCPKG_COMMAND and VCPKG_X_RECURSIVE_DATA. Child processes can call vcpkg via "$VCPKG_COMMAND <args>" in limited internal circumstances.
* [vcpkg] Address CR comments
* [vcpkg] Do not move through Optional<&> into Optional<T>
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
* [vcpkg] add env var VCPKG_OVERLAY_TRIPLETS
* shorten way to complicated doc
* run clang format
* Apply CR
Co-authored-by: ras0219 <533828+ras0219@users.noreply.github.com>
* Apply CR2
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
Co-authored-by: ras0219 <533828+ras0219@users.noreply.github.com>
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
Co-authored-by: Billy Robert O'Neal III <bion@microsoft.com>
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
* Add compiler info to nuspec description
* Run clang-format on some files
* Fix the unit tests
* [vcpkg] Clarify NuGet description to note 'CXX compiler'
* [vcpkg] Fix tests
Co-authored-by: frivard <frivard@coveo.com>
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
* Fix the case of current_path() before use on Windows.
This is a better solution than that I tried in https://github.com/microsoft/vcpkg/pull/13144 -- rather than trying to cannoicalize the path completely, which would destroy symlinks etc; this form calls FindFirstFile on each path element to get the real case from the filesystem.
Fixes#13105.
(Note the wrong case in the CWD in the prompt, but the correct case in all the output:)
PS C:\DeV\vcPKG> .\vcpkg.exe install curl
Computing installation plan...
The following packages will be built and installed:
curl[core,non-http,schannel,ssl,sspi,winssl]:x86-windows
* zlib[core]:x86-windows
Additional packages (*) will be modified to complete this operation.
Detecting compiler hash for triplet x86-windows...
Starting package 1/2: zlib:x86-windows
Building package zlib[core]:x86-windows...
Using cached binary package: C:\Users\billy\AppData\Local\vcpkg/archives\c6\c61dd1bcc23348934c55f4ced77f1e4b0f353394.zipBuilding package zlib[core]:x86-windows... done
Installing package zlib[core]:x86-windows...
Installing package zlib[core]:x86-windows... done
Elapsed time for package zlib:x86-windows: 62.26 ms
Starting package 2/2: curl:x86-windows
Building package curl[core,non-http,schannel,ssl,sspi,winssl]:x86-windows...
Could not locate cached archive: C:\Users\billy\AppData\Local\vcpkg/archives\9f\9fa16d473c9539e9ea7ab3922eff46618f3a4c4b.zip
-- Downloading 9d954e49bc.tar.gz...
-- Extracting source C:/Dev/vcpkg/downloads/curl-curl-9d954e49bce3706a9a2efb119ecd05767f0f2a9e.tar.gz
-- Applying patch 0002_fix_uwp.patch
-- Applying patch 0004_nghttp2_staticlib.patch
-- Applying patch 0005_remove_imp_suffix.patch
-- Applying patch 0006_fix_tool_depends.patch
-- Applying patch 0007_disable_tool_export_curl_target.patch
-- Applying patch 0009_fix_openssl_config.patch
-- Applying patch 0010_fix_othertests_cmake.patch
-- Applying patch 0011_fix_static_build.patch
-- Using source at C:/Dev/vcpkg/buildtrees/curl/src/767f0f2a9e-91d24adee1.clean
-- Configuring x86-windows
-- Building x86-windows-dbg
-- Building x86-windows-rel
-- Installing: C:/Dev/vcpkg/packages/curl_x86-windows/share/curl/curl-config
-- Installing: C:/Dev/vcpkg/packages/curl_x86-windows/share/curl/vcpkg-cmake-wrapper.cmake
-- Installing: C:/Dev/vcpkg/packages/curl_x86-windows/share/curl/copyright
-- Performing post-build validation
-- Performing post-build validation done
Stored binary cache: C:\Users\billy\AppData\Local\vcpkg/archives\9f\9fa16d473c9539e9ea7ab3922eff46618f3a4c4b.zip
Building package curl[core,non-http,schannel,ssl,sspi,winssl]:x86-windows... done
Installing package curl[core,non-http,schannel,ssl,sspi,winssl]:x86-windows...
Installing package curl[core,non-http,schannel,ssl,sspi,winssl]:x86-windows... done
Elapsed time for package curl:x86-windows: 1.299 min
Total elapsed time: 1.33 min
The package curl:x86-windows provides CMake targets:
find_package(CURL CONFIG REQUIRED)
target_link_libraries(main PRIVATE CURL::libcurl)
PS C:\DeV\vcPKG>
* Fix *nix builds.
* PR feedback.
* Update toolsrc/src/vcpkg/base/files.cpp
Co-authored-by: ras0219 <533828+ras0219@users.noreply.github.com>
* [vcpkg docs] add docs for manifest files
These are just for the maintainer docs,
not user docs.
* [vcpkg] EBNF-ify platform expression parsing
this modifies nothing about what strings are accepted or rejected,
it just moves stuff around.
also adds tests.
* [vcpkg docs] add manifest mode example
* [wip] docs for augustin
also fix tabs
* [vcpkg manifest] switch to using maps for features
* Apply suggestions from code review
* un-experimentize format-manifest
* flesh out the user manifest mode docs
* CRs
* billy CRs
* final personal pass-thru
* [vcpkg] Fix error reporting on json parse failure
* [vcpkg] Track manifest path for use in diagnostics
* [vcpkg] Use by-value for consumer API. Improve trailing comma diagnostic.
* [vcpkg] Track errors directly inside Json::Reader
* [vcpkg] Fixup use of .u8string()
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
This tag is required to correctly interface with GitHub Packages, because GHP shares a single remote for the entire organization and uses the repository field to associate a package with a particular repository.
This tag will be automatically generated if the default GitHub Actions environment variables are present (GITHUB_XYZ) or if the user defines VCPKG_NUGET_REPOSITORY.
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
