diff --git a/docs/specifications/ports-overlay.md b/docs/specifications/ports-overlay.md new file mode 100644 index 0000000000..e877f50109 --- /dev/null +++ b/docs/specifications/ports-overlay.md @@ -0,0 +1,178 @@ +# Ports Overlay (Jun 19, 2019) + + +## 1. Motivation + +### A. Allow users to override ports with alternate versions + +It's a common scenario for `vcpkg` users to keep specific versions of libraries to use in their own projects. The current recommendation for users is to fork `vcpkg`'s repository and create tags for commits containing the specific versions of the ports they want to use. + +This proposal adds an alternative to solve this problem. By allowing `vcpkg` users to specify additional locations in their file system containing ports for: + + * older or newer versions of libraries, + * modified libraries, or + * libraries not available in `vcpkg`. + +These locations will be searched when resolving port names during package installation, and override ports in `/ports`. + +### B. Allow users to keep unmodified upstream ports + +Users will be able to keep unmodified versions of the ports shipped with `vcpkg` and update them via `vcpkg update` and `vcpkg upgrade` without having to solve merge conflicts. + + +## 2. Other design concerns + +* Allow a set of `vcpkg` commands to optionally accept additional paths to be used when searching for ports. +* Additional paths must take precedence when resolving names of ports to install. +* Allow users to specify multiple additional paths. +* Provide a simple disambiguation mechanism to resolve ambiguous port names. +* After resolving a port name, the installation process has to work the same as for ports shipped by `vcpkg`. +* This **DOES NOT ENABLE MULTIPLE VERSIONS** of a same library to be **INSTALLED SIDE-BY-SIDE**. + + +## 3. Proposed solution + +This document proposes allowing additional locations to search for ports during package installation that will override and complement the set of ports provided by `vcpkg` (ports under the `/ports` directory).` + +A new option `--overlay-ports` will be added to the `vcpkg install`, `vcpkg update`, `vcpkg upgrade`, `vcpkg export`, and `vcpkg depend-info` commands to specify additional paths containing ports. + +From a user experience perspective, a user expresses interest in adding additional lookup locations by passing the `--overlay-ports` option followed by a path to: + +* an individual port (directory containing a `CONTROL` file), + * `vcpkg install sqlite3 --overlay-ports="C:\custom-ports\sqlite3"` + +* a directory containing ports, + * `vcpkg install sqlite3 --overlay-ports=\\share\org\custom-ports` + +* a file listing paths to the former two. + * `vcpkg install sqlite3 --overlay-ports=..\port-repos.txt` + + _port-repos.txt_ + + ``` + .\experimental-ports\sqlite3 + C:\custom-ports + \\share\team\custom-ports + \\share\org\custom-ports + ``` + *Relative paths inside this file are resolved relatively to the file's location. In this case a `experimental-ports` directory should exist at the same level as the `port-repos.txt` file.* + +_NOTE: It is not the goal of this document to discuss library versioning or project dependency management solutions, which require the ability to install multiple versions of a same library side-by-side._ + +### Multiple additional paths + +Users can provide multiple additional paths by repeating the `--overlay-ports` option. + +``` +vcpkg install sqlite3 + --overlay-ports="..\experimental-ports\sqlite3" + --overlay-ports="C:\custom-ports" + --overlay-ports="\\share\team\custom-ports +``` + +### Overlaying ports + +Port name resolution follows the order in which additional paths are specified, with the first match being selected for installation, and falling back to `/ports` if the port is not found in any of the additional paths. + +No effort is made to compare version numbers inside the `CONTROL` files, or to determine which port contains newer or older files. + +### Examples + +Given the following directory structure: + + ``` + team-ports/ + |-- sqlite3/ + |---- CONTROL + |-- rapidjson/ + |---- CONTROL + |-- curl/ + |---- CONTROL + + my-ports/ + |-- sqlite3/ + |---- CONTROL + |-- rapidjson/ + |---- CONTROL + + vcpkg + |-- ports/ + |---- + |-- vcpkg.exe + |-- preferred-ports.txt + ``` +* #### Example #1: + + Running: + + ``` + vcpkg/vcpkg.exe install sqlite3 --overlay-ports=my-ports --overlay-ports=team-ports + ``` + + Results in `my-ports/sqlite3` getting installed as that location appears first in the command line arguments. + +* #### Example #2: + + A specific version of a port can be given priority by adding its path first in the list of arguments: + + ``` + vcpkg/vcpkg.exe install sqlite3 rapidjson curl + --overlay-ports=my-ports/rapidjson + --overlay-ports=vcpkg/ports/curl + --overlay-ports=team-ports + ``` + + Installs: + * `sqlite3` from `team-ports/sqlite3` + * `rapidjson` from `my-ports/rapidjson` + * `curl` from `vcpkg/ports/curl` + +* #### Example #3: + + Given the content of `preferred-ports.txt` as: + + ``` + ./ports/curl + /my-ports/rapidjson + /team-ports + ``` + + A location can be appended or prepended to those included in `preferred-ports.txt` via the command line, like this: + + ``` + vcpkg/vcpkg.exe install sqlite3 curl --overlay-ports=my-ports --overlay-ports=vcpkg/preferred-ports.txt + ``` + + Which results in `my-ports/sqlite3` and `vcpkg/ports/curl` getting installed. + + +## 4. Proposed User experience + +### i. User wants to preserve an older version of a port + +Developer Alice and her team use `vcpkg` to acquire **OpenCV** and some other packages. She has even contributed many patches to add features to the **OpenCV 3** port in `vcpkg`. But, one day, she notices that a PR to update **OpenCV** to the next major version has been merged. + +Alice wants to update some packages available in `vcpkg`. Unfortunately, updating her project to use the latest **OpenCV** is not immediately possible. + +Alice creates a private GitHub repository and checks in the set of ports that she wants to preserve. Then provides her teammates with the link to clone her private ports repository. + +``` +mkdir vcpkg-custom-ports +cd vcpkg-custom-ports +git init +cp -r %VCPKG_ROOT%/ports/opencv . +git add . +git commit -m "[opencv] Add OpenCV 3 port" +git remote add origin https://github.com//vcpkg-custom-ports.git +git push -u origin master +``` + +Now her team is able to use: + +``` +git clone https://github.com//vcpkg-custom-ports.git +vcpkg update --overlay-ports=./vcpkg-custom-ports +vcpkg upgrade --no-dry-run --overlay-ports=./vcpkg-custom-ports +``` + +to upgrade their packages and preserve the old version of **OpenCV** they require. diff --git a/toolsrc/include/vcpkg/build.h b/toolsrc/include/vcpkg/build.h index 9044cb5566..04cd7cf873 100644 --- a/toolsrc/include/vcpkg/build.h +++ b/toolsrc/include/vcpkg/build.h @@ -20,7 +20,7 @@ namespace vcpkg::Build namespace Command { void perform_and_exit_ex(const FullPackageSpec& full_spec, - const fs::path& port_dir, + const SourceControlFileLocation& scfl, const ParsedArguments& options, const VcpkgPaths& paths); diff --git a/toolsrc/include/vcpkg/dependencies.h b/toolsrc/include/vcpkg/dependencies.h index 16fdb3f73d..8c2050b3d1 100644 --- a/toolsrc/include/vcpkg/dependencies.h +++ b/toolsrc/include/vcpkg/dependencies.h @@ -48,7 +48,7 @@ namespace vcpkg::Dependencies const RequestType& request_type); InstallPlanAction(const PackageSpec& spec, - const SourceControlFile& scf, + const SourceControlFileLocation& scfl, const std::set& features, const RequestType& request_type, std::vector&& dependencies); @@ -57,7 +57,7 @@ namespace vcpkg::Dependencies PackageSpec spec; - Optional source_control_file; + Optional source_control_file_location; Optional installed_package; InstallPlanType plan_type; @@ -129,26 +129,31 @@ namespace vcpkg::Dependencies struct PortFileProvider { - virtual Optional get_control_file(const std::string& src_name) const = 0; + virtual Optional get_control_file(const std::string& src_name) const = 0; + virtual std::vector load_all_control_files() const = 0; }; struct MapPortFileProvider : Util::ResourceBase, PortFileProvider { - explicit MapPortFileProvider(const std::unordered_map& map); - Optional get_control_file(const std::string& src_name) const override; + explicit MapPortFileProvider(const std::unordered_map& map); + Optional get_control_file(const std::string& src_name) const override; + std::vector load_all_control_files() const override; private: - const std::unordered_map& ports; + const std::unordered_map& ports; }; struct PathsPortFileProvider : Util::ResourceBase, PortFileProvider { - explicit PathsPortFileProvider(const VcpkgPaths& paths); - Optional get_control_file(const std::string& src_name) const override; + explicit PathsPortFileProvider(const vcpkg::VcpkgPaths& paths, + const std::vector* ports_dirs_paths); + Optional get_control_file(const std::string& src_name) const override; + std::vector load_all_control_files() const override; private: - const VcpkgPaths& ports; - mutable std::unordered_map cache; + Files::Filesystem& filesystem; + std::vector ports_dirs; + mutable std::unordered_map cache; }; struct ClusterGraph; @@ -181,7 +186,7 @@ namespace vcpkg::Dependencies std::vector create_export_plan(const std::vector& specs, const StatusParagraphs& status_db); - std::vector create_feature_install_plan(const std::unordered_map& map, + std::vector create_feature_install_plan(const std::unordered_map& map, const std::vector& specs, const StatusParagraphs& status_db); diff --git a/toolsrc/include/vcpkg/postbuildlint.h b/toolsrc/include/vcpkg/postbuildlint.h index 5dcfeb8dfa..027619eb8b 100644 --- a/toolsrc/include/vcpkg/postbuildlint.h +++ b/toolsrc/include/vcpkg/postbuildlint.h @@ -9,5 +9,6 @@ namespace vcpkg::PostBuildLint size_t perform_all_checks(const PackageSpec& spec, const VcpkgPaths& paths, const Build::PreBuildInfo& pre_build_info, - const Build::BuildInfo& build_info); + const Build::BuildInfo& build_info, + const fs::path& port_dir); } diff --git a/toolsrc/include/vcpkg/sourceparagraph.h b/toolsrc/include/vcpkg/sourceparagraph.h index d70fd43373..6232a3fd29 100644 --- a/toolsrc/include/vcpkg/sourceparagraph.h +++ b/toolsrc/include/vcpkg/sourceparagraph.h @@ -68,6 +68,15 @@ namespace vcpkg Optional find_feature(const std::string& featurename) const; }; + /// + /// Full metadata of a package: core and other features. As well as the location the SourceControlFile was loaded from. + /// + struct SourceControlFileLocation + { + std::unique_ptr source_control_file; + fs::path source_location; + }; + void print_error_message(Span> error_info_list); inline void print_error_message(const std::unique_ptr& error_info_list) { diff --git a/toolsrc/include/vcpkg/vcpkgcmdarguments.h b/toolsrc/include/vcpkg/vcpkgcmdarguments.h index de65eec286..cad013eb88 100644 --- a/toolsrc/include/vcpkg/vcpkgcmdarguments.h +++ b/toolsrc/include/vcpkg/vcpkgcmdarguments.h @@ -15,6 +15,7 @@ namespace vcpkg { std::unordered_set switches; std::unordered_map settings; + std::unordered_map> multisettings; }; struct VcpkgPaths; @@ -41,10 +42,22 @@ namespace vcpkg StringLiteral short_help_text; }; + struct CommandMultiSetting + { + constexpr CommandMultiSetting(const StringLiteral& name, const StringLiteral& short_help_text) + : name(name), short_help_text(short_help_text) + { + } + + StringLiteral name; + StringLiteral short_help_text; + }; + struct CommandOptionsStructure { Span switches; Span settings; + Span multisettings; }; struct CommandStructure @@ -74,6 +87,7 @@ namespace vcpkg std::unique_ptr vcpkg_root_dir; std::unique_ptr triplet; + std::unique_ptr> overlay_ports; Optional debug = nullopt; Optional sendmetrics = nullopt; Optional printmetrics = nullopt; @@ -88,6 +102,6 @@ namespace vcpkg ParsedArguments parse_arguments(const CommandStructure& command_structure) const; private: - std::unordered_map> optional_command_arguments; + std::unordered_map>> optional_command_arguments; }; } diff --git a/toolsrc/include/vcpkg/vcpkgpaths.h b/toolsrc/include/vcpkg/vcpkgpaths.h index 42de40d9c0..b09169b027 100644 --- a/toolsrc/include/vcpkg/vcpkgpaths.h +++ b/toolsrc/include/vcpkg/vcpkgpaths.h @@ -50,8 +50,6 @@ namespace vcpkg static Expected create(const fs::path& vcpkg_root_dir, const std::string& default_vs_path); fs::path package_dir(const PackageSpec& spec) const; - fs::path port_dir(const PackageSpec& spec) const; - fs::path port_dir(const std::string& name) const; fs::path build_info_file_path(const PackageSpec& spec) const; fs::path listfile_path(const BinaryParagraph& pgh) const; diff --git a/toolsrc/src/tests.plan.cpp b/toolsrc/src/tests.plan.cpp index 238aa70324..ab24266b48 100644 --- a/toolsrc/src/tests.plan.cpp +++ b/toolsrc/src/tests.plan.cpp @@ -47,7 +47,8 @@ namespace UnitTest1 Assert::AreEqual(plan.spec.triplet().to_string().c_str(), triplet.to_string().c_str()); - Assert::AreEqual(pkg_name.c_str(), plan.source_control_file.get()->core_paragraph->name.c_str()); + auto* scfl = plan.source_control_file_location.get(); + Assert::AreEqual(pkg_name.c_str(), scfl->source_control_file->core_paragraph->name.c_str()); Assert::AreEqual(size_t(vec.size()), feature_list.size()); for (auto&& feature_name : vec) @@ -79,7 +80,7 @@ namespace UnitTest1 /// struct PackageSpecMap { - std::unordered_map map; + std::unordered_map map; Triplet triplet; PackageSpecMap(const Triplet& t = Triplet::X86_WINDOWS) noexcept { triplet = t; } @@ -94,7 +95,8 @@ namespace UnitTest1 { auto spec = PackageSpec::from_name_and_triplet(scf.core_paragraph->name, triplet); Assert::IsTrue(spec.has_value()); - map.emplace(scf.core_paragraph->name, std::move(scf)); + map.emplace(scf.core_paragraph->name, + SourceControlFileLocation{std::unique_ptr(std::move(&scf)), ""}); return PackageSpec{*spec.get()}; } }; diff --git a/toolsrc/src/tests.update.cpp b/toolsrc/src/tests.update.cpp index b6e487c17f..5e3f9f3e20 100644 --- a/toolsrc/src/tests.update.cpp +++ b/toolsrc/src/tests.update.cpp @@ -21,9 +21,9 @@ namespace UnitTest1 StatusParagraphs status_db(std::move(status_paragraphs)); - std::unordered_map map; + std::unordered_map map; auto scf = unwrap(SourceControlFile::parse_control_file(Pgh{{{"Source", "a"}, {"Version", "0"}}})); - map.emplace("a", std::move(*scf)); + map.emplace("a", SourceControlFileLocation { std::move(scf), "" }); Dependencies::MapPortFileProvider provider(map); auto pkgs = SortedVector(Update::find_outdated_packages(provider, status_db), @@ -45,9 +45,9 @@ namespace UnitTest1 StatusParagraphs status_db(std::move(status_paragraphs)); - std::unordered_map map; + std::unordered_map map; auto scf = unwrap(SourceControlFile::parse_control_file(Pgh{{{"Source", "a"}, {"Version", "0"}}})); - map.emplace("a", std::move(*scf)); + map.emplace("a", SourceControlFileLocation { std::move(scf), "" }); Dependencies::MapPortFileProvider provider(map); auto pkgs = SortedVector(Update::find_outdated_packages(provider, status_db), @@ -71,9 +71,9 @@ namespace UnitTest1 StatusParagraphs status_db(std::move(status_paragraphs)); - std::unordered_map map; + std::unordered_map map; auto scf = unwrap(SourceControlFile::parse_control_file(Pgh{{{"Source", "a"}, {"Version", "0"}}})); - map.emplace("a", std::move(*scf)); + map.emplace("a", SourceControlFileLocation{ std::move(scf), "" }); Dependencies::MapPortFileProvider provider(map); auto pkgs = SortedVector(Update::find_outdated_packages(provider, status_db), @@ -92,9 +92,9 @@ namespace UnitTest1 StatusParagraphs status_db(std::move(status_paragraphs)); - std::unordered_map map; + std::unordered_map map; auto scf = unwrap(SourceControlFile::parse_control_file(Pgh{{{"Source", "a"}, {"Version", "2"}}})); - map.emplace("a", std::move(*scf)); + map.emplace("a", SourceControlFileLocation{ std::move(scf), "" }); Dependencies::MapPortFileProvider provider(map); auto pkgs = SortedVector(Update::find_outdated_packages(provider, status_db), diff --git a/toolsrc/src/vcpkg/build.cpp b/toolsrc/src/vcpkg/build.cpp index daff7e6c8f..059a09432c 100644 --- a/toolsrc/src/vcpkg/build.cpp +++ b/toolsrc/src/vcpkg/build.cpp @@ -23,6 +23,7 @@ #include using vcpkg::Build::BuildResult; +using vcpkg::Dependencies::PathsPortFileProvider; using vcpkg::Parse::ParseControlErrorInfo; using vcpkg::Parse::ParseExpected; @@ -34,34 +35,26 @@ namespace vcpkg::Build::Command static constexpr StringLiteral OPTION_CHECKS_ONLY = "--checks-only"; void perform_and_exit_ex(const FullPackageSpec& full_spec, - const fs::path& port_dir, + const SourceControlFileLocation& scfl, const ParsedArguments& options, const VcpkgPaths& paths) { const PackageSpec& spec = full_spec.package_spec; + const auto& scf = *scfl.source_control_file; if (Util::Sets::contains(options.switches, OPTION_CHECKS_ONLY)) { const auto pre_build_info = Build::PreBuildInfo::from_triplet_file(paths, spec.triplet()); const auto build_info = Build::read_build_info(paths.get_filesystem(), paths.build_info_file_path(spec)); - const size_t error_count = PostBuildLint::perform_all_checks(spec, paths, pre_build_info, build_info); + const size_t error_count = + PostBuildLint::perform_all_checks(spec, paths, pre_build_info, build_info, scfl.source_location); Checks::check_exit(VCPKG_LINE_INFO, error_count == 0); Checks::exit_success(VCPKG_LINE_INFO); } - const ParseExpected source_control_file = - Paragraphs::try_load_port(paths.get_filesystem(), port_dir); - - if (!source_control_file.has_value()) - { - print_error_message(source_control_file.error()); - Checks::exit_fail(VCPKG_LINE_INFO); - } - - const auto& scf = source_control_file.value_or_exit(VCPKG_LINE_INFO); Checks::check_exit(VCPKG_LINE_INFO, - spec.name() == scf->core_paragraph->name, + spec.name() == scf.core_paragraph->name, "The Source field inside the CONTROL file does not match the port directory: '%s' != '%s'", - scf->core_paragraph->name, + scf.core_paragraph->name, spec.name()); const StatusParagraphs status_db = database_load_check(paths); @@ -80,7 +73,7 @@ namespace vcpkg::Build::Command features_as_set.emplace("core"); const Build::BuildPackageConfig build_config{ - *scf, spec.triplet(), fs::path{port_dir}, build_package_options, features_as_set}; + scf, spec.triplet(), fs::path(scfl.source_location), build_package_options, features_as_set}; const auto build_timer = Chrono::ElapsedTimer::create_started(); const auto result = Build::build_package(paths, build_config, status_db); @@ -128,10 +121,19 @@ namespace vcpkg::Build::Command // Build only takes a single package and all dependencies must already be installed const ParsedArguments options = args.parse_arguments(COMMAND_STRUCTURE); std::string first_arg = args.command_arguments.at(0); + const FullPackageSpec spec = Input::check_and_get_full_package_spec( std::move(first_arg), default_triplet, COMMAND_STRUCTURE.example_text); + Input::check_triplet(spec.package_spec.triplet(), paths); - perform_and_exit_ex(spec, paths.port_dir(spec.package_spec), options, paths); + + PathsPortFileProvider provider(paths, args.overlay_ports.get()); + const auto port_name = spec.package_spec.name(); + const auto* scfl = provider.get_control_file(port_name).get(); + + Checks::check_exit(VCPKG_LINE_INFO, scfl != nullptr, "Error: Couldn't find port '%s'", port_name); + + perform_and_exit_ex(spec, *scfl, options, paths); } } @@ -360,6 +362,12 @@ namespace vcpkg::Build auto& fs = paths.get_filesystem(); const Triplet& triplet = spec.triplet(); + if (!Strings::starts_with(Strings::ascii_to_lowercase(config.port_dir.u8string()), + Strings::ascii_to_lowercase(paths.ports.u8string()))) + { + System::printf("-- Installing port from location: %s\n", config.port_dir.u8string()); + } + #if !defined(_WIN32) // TODO: remove when vcpkg.exe is in charge for acquiring tools. Change introduced in vcpkg v0.0.107. // bootstrap should have already downloaded ninja, but making sure it is present in case it was deleted. @@ -428,7 +436,8 @@ namespace vcpkg::Build } const BuildInfo build_info = read_build_info(fs, paths.build_info_file_path(spec)); - const size_t error_count = PostBuildLint::perform_all_checks(spec, paths, pre_build_info, build_info); + const size_t error_count = + PostBuildLint::perform_all_checks(spec, paths, pre_build_info, build_info, config.port_dir); auto bcf = create_binary_control_file(*config.scf.core_paragraph, triplet, build_info, abi_tag); @@ -705,7 +714,7 @@ namespace vcpkg::Build } } - System::print2("Could not locate cached archive: ", archive_path.u8string(), "\n"); + System::printf("Could not locate cached archive: %s\n", archive_path.u8string()); ExtendedBuildResult result = do_build_package_and_clean_buildtrees( paths, pre_build_info, spec, maybe_abi_tag_and_file.value_or(AbiTagAndFile{}).tag, config); diff --git a/toolsrc/src/vcpkg/commands.autocomplete.cpp b/toolsrc/src/vcpkg/commands.autocomplete.cpp index afef518eb0..3cf4dd98fd 100644 --- a/toolsrc/src/vcpkg/commands.autocomplete.cpp +++ b/toolsrc/src/vcpkg/commands.autocomplete.cpp @@ -90,7 +90,8 @@ namespace vcpkg::Commands::Autocomplete const auto port_name = match[2].str(); const auto triplet_prefix = match[3].str(); - auto maybe_port = Paragraphs::try_load_port(paths.get_filesystem(), paths.port_dir(port_name)); + // TODO: Support autocomplete for ports in --overlay-ports + auto maybe_port = Paragraphs::try_load_port(paths.get_filesystem(), paths.ports / port_name); if (maybe_port.error()) { Checks::exit_success(VCPKG_LINE_INFO); diff --git a/toolsrc/src/vcpkg/commands.buildexternal.cpp b/toolsrc/src/vcpkg/commands.buildexternal.cpp index 19b89c2f52..1c3c511c27 100644 --- a/toolsrc/src/vcpkg/commands.buildexternal.cpp +++ b/toolsrc/src/vcpkg/commands.buildexternal.cpp @@ -23,7 +23,12 @@ namespace vcpkg::Commands::BuildExternal std::string(args.command_arguments.at(0)), default_triplet, COMMAND_STRUCTURE.example_text); Input::check_triplet(spec.package_spec.triplet(), paths); - const fs::path port_dir = args.command_arguments.at(1); - Build::Command::perform_and_exit_ex(spec, port_dir, options, paths); + auto overlays = args.overlay_ports ? *args.overlay_ports : std::vector(); + overlays.insert(overlays.begin(), args.command_arguments.at(1)); + Dependencies::PathsPortFileProvider provider(paths, &overlays); + auto maybe_scfl = provider.get_control_file(spec.package_spec.name()); + Checks::check_exit( + VCPKG_LINE_INFO, maybe_scfl.has_value(), "could not load control file for %s", spec.package_spec.name()); + Build::Command::perform_and_exit_ex(spec, maybe_scfl.value_or_exit(VCPKG_LINE_INFO), options, paths); } } diff --git a/toolsrc/src/vcpkg/commands.ci.cpp b/toolsrc/src/vcpkg/commands.ci.cpp index de66d917b3..8cab79504a 100644 --- a/toolsrc/src/vcpkg/commands.ci.cpp +++ b/toolsrc/src/vcpkg/commands.ci.cpp @@ -232,12 +232,17 @@ namespace vcpkg::Commands::CI { // determine abi tag std::string abi; - if (auto scf = p->source_control_file.get()) + if (auto scfl = p->source_control_file_location.get()) { auto triplet = p->spec.triplet(); const Build::BuildPackageConfig build_config{ - *scf, triplet, paths.port_dir(p->spec), build_options, p->feature_list}; + *scfl->source_control_file, + triplet, + static_cast(scfl->source_location), + build_options, + p->feature_list + }; auto dependency_abis = Util::fmap(p->computed_dependencies, [&](const PackageSpec& spec) -> Build::AbiEntry { @@ -351,7 +356,8 @@ namespace vcpkg::Commands::CI } StatusParagraphs status_db = database_load_check(paths); - const auto& paths_port_file = Dependencies::PathsPortFileProvider(paths); + + Dependencies::PathsPortFileProvider provider(paths, args.overlay_ports.get()); const Build::BuildPackageOptions install_plan_options = { Build::UseHeadVersion::NO, @@ -369,7 +375,10 @@ namespace vcpkg::Commands::CI XunitTestResults xunitTestResults; - std::vector all_ports = Install::get_all_port_names(paths); + std::vector all_ports = + Util::fmap(provider.load_all_control_files(), [](auto&& scfl) -> std::string { + return scfl->source_control_file.get()->core_paragraph->name; + }); std::vector results; auto timer = Chrono::ElapsedTimer::create_started(); for (const Triplet& triplet : triplets) @@ -378,13 +387,13 @@ namespace vcpkg::Commands::CI xunitTestResults.push_collection(triplet.canonical_name()); - Dependencies::PackageGraph pgraph(paths_port_file, status_db); + Dependencies::PackageGraph pgraph(provider, status_db); std::vector specs = PackageSpec::to_package_specs(all_ports, triplet); // Install the default features for every package auto all_feature_specs = Util::fmap(specs, [](auto& spec) { return FeatureSpec(spec, ""); }); auto split_specs = - find_unknown_ports_for_ci(paths, exclusions_set, paths_port_file, all_feature_specs, purge_tombstones); + find_unknown_ports_for_ci(paths, exclusions_set, provider, all_feature_specs, purge_tombstones); auto feature_specs = FullPackageSpec::to_feature_specs(split_specs->unknown); for (auto&& fspec : feature_specs) diff --git a/toolsrc/src/vcpkg/commands.dependinfo.cpp b/toolsrc/src/vcpkg/commands.dependinfo.cpp index 48a289fa5d..c0800a4b54 100644 --- a/toolsrc/src/vcpkg/commands.dependinfo.cpp +++ b/toolsrc/src/vcpkg/commands.dependinfo.cpp @@ -6,6 +6,9 @@ #include #include #include +#include + +using vcpkg::Dependencies::PathsPortFileProvider; namespace vcpkg::Commands::DependInfo { @@ -35,7 +38,7 @@ namespace vcpkg::Commands::DependInfo return output; } - std::string create_dot_as_string(const std::vector>& source_control_files) + std::string create_dot_as_string(const std::vector& source_control_files) { int empty_node_count = 0; @@ -64,7 +67,7 @@ namespace vcpkg::Commands::DependInfo return s; } - std::string create_dgml_as_string(const std::vector>& source_control_files) + std::string create_dgml_as_string(const std::vector& source_control_files) { std::string s; s.append(""); @@ -109,7 +112,7 @@ namespace vcpkg::Commands::DependInfo } std::string create_graph_as_string(const std::unordered_set& switches, - const std::vector>& source_control_files) + const std::vector& source_control_files) { if (Util::Sets::contains(switches, OPTION_DOT)) { @@ -124,7 +127,7 @@ namespace vcpkg::Commands::DependInfo void build_dependencies_list(std::set& packages_to_keep, const std::string& requested_package, - const std::vector>& source_control_files, + const std::vector& source_control_files, const std::unordered_set& switches) { const auto source_control_file = @@ -154,7 +157,11 @@ namespace vcpkg::Commands::DependInfo { const ParsedArguments options = args.parse_arguments(COMMAND_STRUCTURE); - auto source_control_files = Paragraphs::load_all_ports(paths.get_filesystem(), paths.ports); + // TODO: Optimize implementation, current implementation needs to load all ports from disk which is too slow. + PathsPortFileProvider provider(paths, args.overlay_ports.get()); + auto source_control_files = Util::fmap(provider.load_all_control_files(), [](auto&& scfl) -> const SourceControlFile * { + return scfl->source_control_file.get(); + }); if (args.command_arguments.size() >= 1) { @@ -178,7 +185,7 @@ namespace vcpkg::Commands::DependInfo for (auto&& source_control_file : source_control_files) { - const SourceParagraph& source_paragraph = *source_control_file->core_paragraph; + const SourceParagraph& source_paragraph = *source_control_file->core_paragraph.get(); const auto s = Strings::join(", ", source_paragraph.depends, [](const Dependency& d) { return d.name(); }); System::print2(source_paragraph.name, ": ", s, "\n"); } diff --git a/toolsrc/src/vcpkg/commands.edit.cpp b/toolsrc/src/vcpkg/commands.edit.cpp index 98176b2b05..f2f0b15693 100644 --- a/toolsrc/src/vcpkg/commands.edit.cpp +++ b/toolsrc/src/vcpkg/commands.edit.cpp @@ -79,6 +79,7 @@ namespace vcpkg::Commands::Edit const auto& fs = paths.get_filesystem(); auto packages = fs.get_files_non_recursive(paths.packages); + // TODO: Support edit for --overlay-ports return Util::fmap(ports, [&](const std::string& port_name) -> std::string { const auto portpath = paths.ports / port_name; const auto portfile = portpath / "portfile.cmake"; diff --git a/toolsrc/src/vcpkg/commands.search.cpp b/toolsrc/src/vcpkg/commands.search.cpp index a0afd69e13..3d8387ee12 100644 --- a/toolsrc/src/vcpkg/commands.search.cpp +++ b/toolsrc/src/vcpkg/commands.search.cpp @@ -7,6 +7,9 @@ #include #include #include +#include + +using vcpkg::Dependencies::PathsPortFileProvider; namespace vcpkg::Commands::Search { @@ -63,7 +66,10 @@ namespace vcpkg::Commands::Search const ParsedArguments options = args.parse_arguments(COMMAND_STRUCTURE); const bool full_description = Util::Sets::contains(options.switches, OPTION_FULLDESC); - auto source_paragraphs = Paragraphs::load_all_ports(paths.get_filesystem(), paths.ports); + PathsPortFileProvider provider(paths, args.overlay_ports.get()); + auto source_paragraphs = Util::fmap(provider.load_all_control_files(), [](auto&& port) -> const SourceControlFile * { + return port->source_control_file.get(); + }); if (args.command_arguments.empty()) { diff --git a/toolsrc/src/vcpkg/commands.upgrade.cpp b/toolsrc/src/vcpkg/commands.upgrade.cpp index 29815ca949..77183ceaf7 100644 --- a/toolsrc/src/vcpkg/commands.upgrade.cpp +++ b/toolsrc/src/vcpkg/commands.upgrade.cpp @@ -43,7 +43,8 @@ namespace vcpkg::Commands::Upgrade StatusParagraphs status_db = database_load_check(paths); - Dependencies::PathsPortFileProvider provider(paths); + // Load ports from ports dirs + Dependencies::PathsPortFileProvider provider(paths, args.overlay_ports.get()); Dependencies::PackageGraph graph(provider, status_db); // input sanitization @@ -85,12 +86,12 @@ namespace vcpkg::Commands::Upgrade not_installed.push_back(spec); } - auto maybe_scf = provider.get_control_file(spec.name()); - if (auto p_scf = maybe_scf.get()) + auto maybe_scfl = provider.get_control_file(spec.name()); + if (auto p_scfl = maybe_scfl.get()) { if (it != status_db.end()) { - if (p_scf->core_paragraph->version != (*it)->package.version) + if (p_scfl->source_control_file->core_paragraph->version != (*it)->package.version) { to_upgrade.push_back(spec); } diff --git a/toolsrc/src/vcpkg/dependencies.cpp b/toolsrc/src/vcpkg/dependencies.cpp index f4a05e7dc9..df472515fd 100644 --- a/toolsrc/src/vcpkg/dependencies.cpp +++ b/toolsrc/src/vcpkg/dependencies.cpp @@ -22,7 +22,7 @@ namespace vcpkg::Dependencies struct ClusterSource { - const SourceControlFile* scf = nullptr; + const SourceControlFileLocation* scfl = nullptr; std::unordered_map> build_edges; }; @@ -92,12 +92,12 @@ namespace vcpkg::Dependencies if (it == m_graph.end()) { // Load on-demand from m_provider - auto maybe_scf = m_provider.get_control_file(spec.name()); + auto maybe_scfl = m_provider.get_control_file(spec.name()); auto& clust = m_graph[spec]; clust.spec = spec; - if (auto p_scf = maybe_scf.get()) + if (auto p_scfl = maybe_scfl.get()) { - clust.source = cluster_from_scf(*p_scf, clust.spec.triplet()); + clust.source = cluster_from_scf(*p_scfl, clust.spec.triplet()); } return clust; } @@ -105,15 +105,17 @@ namespace vcpkg::Dependencies } private: - static ClusterSource cluster_from_scf(const SourceControlFile& scf, Triplet t) + static ClusterSource cluster_from_scf(const SourceControlFileLocation& scfl, Triplet t) { ClusterSource ret; - ret.build_edges.emplace("core", filter_dependencies_to_specs(scf.core_paragraph->depends, t)); + ret.build_edges.emplace("core", + filter_dependencies_to_specs(scfl.source_control_file->core_paragraph->depends, + t)); - for (const auto& feature : scf.feature_paragraphs) + for (const auto& feature : scfl.source_control_file->feature_paragraphs) ret.build_edges.emplace(feature->name, filter_dependencies_to_specs(feature->depends, t)); - ret.scf = &scf; + ret.scfl = &scfl; return ret; } @@ -151,12 +153,12 @@ namespace vcpkg::Dependencies } InstallPlanAction::InstallPlanAction(const PackageSpec& spec, - const SourceControlFile& scf, + const SourceControlFileLocation& scfl, const std::set& features, const RequestType& request_type, std::vector&& dependencies) : spec(spec) - , source_control_file(scf) + , source_control_file_location(scfl) , plan_type(InstallPlanType::BUILD_AND_INSTALL) , request_type(request_type) , build_options{} @@ -268,37 +270,145 @@ namespace vcpkg::Dependencies return left->spec.name() < right->spec.name(); } - MapPortFileProvider::MapPortFileProvider(const std::unordered_map& map) : ports(map) - { - } + MapPortFileProvider::MapPortFileProvider(const std::unordered_map& map) + : ports(map) + {} - Optional MapPortFileProvider::get_control_file(const std::string& spec) const + Optional MapPortFileProvider::get_control_file(const std::string& spec) const { auto scf = ports.find(spec); if (scf == ports.end()) return nullopt; return scf->second; } - PathsPortFileProvider::PathsPortFileProvider(const VcpkgPaths& paths) : ports(paths) {} + std::vector MapPortFileProvider::load_all_control_files() const + { + return Util::fmap(ports, [](auto&& kvpair) -> const SourceControlFileLocation * { return &kvpair.second; }); + } - Optional PathsPortFileProvider::get_control_file(const std::string& spec) const + PathsPortFileProvider::PathsPortFileProvider(const vcpkg::VcpkgPaths& paths, + const std::vector* ports_dirs_paths) + : filesystem(paths.get_filesystem()) + { + if (ports_dirs_paths) + { + for (auto&& overlay_path : *ports_dirs_paths) + { + if (!overlay_path.empty()) + { + auto overlay = fs::stdfs::canonical(fs::u8path(overlay_path)); + + Checks::check_exit(VCPKG_LINE_INFO, + filesystem.exists(overlay), + "Error: Path \"%s\" does not exist", + overlay.string()); + + Checks::check_exit(VCPKG_LINE_INFO, + fs::stdfs::is_directory(overlay), + "Error: Path \"%s\" must be a directory", + overlay.string()); + + ports_dirs.emplace_back(overlay); + } + } + } + ports_dirs.emplace_back(paths.ports); + } + + Optional PathsPortFileProvider::get_control_file(const std::string& spec) const { auto cache_it = cache.find(spec); if (cache_it != cache.end()) { return cache_it->second; } - Parse::ParseExpected source_control_file = - Paragraphs::try_load_port(ports.get_filesystem(), ports.port_dir(spec)); - if (auto scf = source_control_file.get()) + for (auto&& ports_dir : ports_dirs) { - auto it = cache.emplace(spec, std::move(*scf->get())); - return it.first->second; + // Try loading individual port + if (filesystem.exists(ports_dir / "CONTROL")) + { + auto maybe_scf = Paragraphs::try_load_port(filesystem, ports_dir); + if (auto scf = maybe_scf.get()) + { + if (scf->get()->core_paragraph->name == spec) + { + SourceControlFileLocation scfl{ std::move(*scf), ports_dir }; + auto it = cache.emplace(spec, std::move(scfl)); + return it.first->second; + } + } + else + { + vcpkg::print_error_message(maybe_scf.error()); + Checks::exit_with_message(VCPKG_LINE_INFO, + "Error: Failed to load port from %s", + spec, ports_dir.u8string()); + } + } + + auto found_scf = Paragraphs::try_load_port(filesystem, ports_dir / spec); + if (auto scf = found_scf.get()) + { + if (scf->get()->core_paragraph->name == spec) + { + SourceControlFileLocation scfl{ std::move(*scf), ports_dir / spec }; + auto it = cache.emplace(spec, std::move(scfl)); + return it.first->second; + } + } } + return nullopt; } + std::vector PathsPortFileProvider::load_all_control_files() const + { + // Reload cache with ports contained in all ports_dirs + cache.clear(); + std::vector ret; + for (auto&& ports_dir : ports_dirs) + { + // Try loading individual port + if (filesystem.exists(ports_dir / "CONTROL")) + { + auto maybe_scf = Paragraphs::try_load_port(filesystem, ports_dir); + if (auto scf = maybe_scf.get()) + { + auto port_name = scf->get()->core_paragraph->name; + if (cache.find(port_name) == cache.end()) + { + SourceControlFileLocation scfl{ std::move(*scf), ports_dir }; + auto it = cache.emplace(port_name, std::move(scfl)); + ret.emplace_back(&it.first->second); + } + } + else + { + vcpkg::print_error_message(maybe_scf.error()); + Checks::exit_with_message(VCPKG_LINE_INFO, + "Error: Failed to load port from %s", + ports_dir.u8string()); + } + continue; + } + + // Try loading all ports inside ports_dir + auto found_scf = Paragraphs::load_all_ports(filesystem, ports_dir); + for (auto&& scf : found_scf) + { + auto port_name = scf->core_paragraph->name; + if (cache.find(port_name) == cache.end()) + { + SourceControlFileLocation scfl{ std::move(scf), ports_dir / port_name }; + auto it = cache.emplace(port_name, std::move(scfl)); + ret.emplace_back(&it.first->second); + } + } + } + return ret; + } + std::vector create_remove_plan(const std::vector& specs, const StatusParagraphs& status_db) { @@ -495,10 +605,11 @@ namespace vcpkg::Dependencies VCPKG_LINE_INFO, "Error: Cannot find definition for package `%s`.", cluster.spec.name()); } + auto&& control_file = *p_source->scfl->source_control_file.get(); if (feature.empty()) { // Add default features for this package. This is an exact reference, so ignore prevent_default_features. - for (auto&& default_feature : p_source->scf->core_paragraph.get()->default_features) + for (auto&& default_feature : control_file.core_paragraph.get()->default_features) { auto res = mark_plus(default_feature, cluster, graph, graph_plan, prevent_default_features); if (res != MarkPlusResult::SUCCESS) @@ -513,7 +624,7 @@ namespace vcpkg::Dependencies if (feature == "*") { - for (auto&& fpgh : p_source->scf->feature_paragraphs) + for (auto&& fpgh : control_file.feature_paragraphs) { auto res = mark_plus(fpgh->name, cluster, graph, graph_plan, prevent_default_features); @@ -590,7 +701,8 @@ namespace vcpkg::Dependencies // Check if any default features have been added auto& previous_df = p_installed->ipv.core->package.default_features; - for (auto&& default_feature : p_source->scf->core_paragraph->default_features) + auto&& control_file = *p_source->scfl->source_control_file.get(); + for (auto&& default_feature : control_file.core_paragraph->default_features) { if (std::find(previous_df.begin(), previous_df.end(), default_feature) == previous_df.end()) { @@ -635,7 +747,7 @@ namespace vcpkg::Dependencies /// Map of all source control files in the current environment. /// Feature specifications to resolve dependencies for. /// Status of installed packages in the current environment. - std::vector create_feature_install_plan(const std::unordered_map& map, + std::vector create_feature_install_plan(const std::unordered_map& map, const std::vector& specs, const StatusParagraphs& status_db) { @@ -698,7 +810,11 @@ namespace vcpkg::Dependencies if (p_cluster->transient_uninstalled) { // If it will be transiently uninstalled, we need to issue a full installation command - auto pscf = p_cluster->source.value_or_exit(VCPKG_LINE_INFO).scf; + auto* pscfl = p_cluster->source.value_or_exit(VCPKG_LINE_INFO).scfl; + Checks::check_exit(VCPKG_LINE_INFO, + pscfl != nullptr, + "Error: Expected a SourceControlFileLocation to exist"); + auto&& scfl = *pscfl; auto dep_specs = Util::fmap(m_graph_plan->install_graph.adjacency_list(p_cluster), [](ClusterPtr const& p) { return p->spec; }); @@ -706,7 +822,7 @@ namespace vcpkg::Dependencies plan.emplace_back(InstallPlanAction{ p_cluster->spec, - *pscf, + scfl, p_cluster->to_install_features, p_cluster->request_type, std::move(dep_specs), diff --git a/toolsrc/src/vcpkg/export.cpp b/toolsrc/src/vcpkg/export.cpp index 23fc7441a6..88c1526c5c 100644 --- a/toolsrc/src/vcpkg/export.cpp +++ b/toolsrc/src/vcpkg/export.cpp @@ -488,7 +488,10 @@ With a project open, go to Tools->NuGet Package Manager->Package Manager Console // create the plan const StatusParagraphs status_db = database_load_check(paths); - Dependencies::PathsPortFileProvider provider(paths); + + // Load ports from ports dirs + Dependencies::PathsPortFileProvider provider(paths, args.overlay_ports.get()); + std::vector export_plan = Dependencies::create_export_plan(opts.specs, status_db); Checks::check_exit(VCPKG_LINE_INFO, !export_plan.empty(), "Export plan cannot be empty"); diff --git a/toolsrc/src/vcpkg/help.cpp b/toolsrc/src/vcpkg/help.cpp index b92e9ca59c..6ee573dfcc 100644 --- a/toolsrc/src/vcpkg/help.cpp +++ b/toolsrc/src/vcpkg/help.cpp @@ -111,10 +111,12 @@ namespace vcpkg::Help " vcpkg contact Display contact information to send feedback\n" "\n" "Options:\n" - " --triplet Specify the target architecture triplet.\n" + " --triplet Specify the target architecture triplet\n" " (default: " ENVVAR(VCPKG_DEFAULT_TRIPLET) // ", see 'vcpkg help triplet')\n" "\n" + " --overlay-ports= Specify directories to be used when searching for ports\n" + "\n" " --vcpkg-root Specify the vcpkg root " "directory\n" " (default: " ENVVAR(VCPKG_ROOT) // diff --git a/toolsrc/src/vcpkg/install.cpp b/toolsrc/src/vcpkg/install.cpp index 88b15fe7a2..781629236a 100644 --- a/toolsrc/src/vcpkg/install.cpp +++ b/toolsrc/src/vcpkg/install.cpp @@ -330,9 +330,10 @@ namespace vcpkg::Install System::printf("Building package %s...\n", display_name_with_features); auto result = [&]() -> Build::ExtendedBuildResult { - const Build::BuildPackageConfig build_config{action.source_control_file.value_or_exit(VCPKG_LINE_INFO), + const auto& scfl = action.source_control_file_location.value_or_exit(VCPKG_LINE_INFO); + const Build::BuildPackageConfig build_config{*scfl.source_control_file, action.spec.triplet(), - paths.port_dir(action.spec), + static_cast(scfl.source_location), action.build_options, action.feature_list}; return Build::build_package(paths, build_config, status_db); @@ -652,13 +653,10 @@ namespace vcpkg::Install Build::FailOnTombstone::NO, }; - auto all_ports = Paragraphs::load_all_ports(fs, paths.ports); - std::unordered_map scf_map; - for (auto&& port : all_ports) - scf_map[port->core_paragraph->name] = std::move(*port); - MapPortFileProvider provider(scf_map); + //// Load ports from ports dirs + PathsPortFileProvider provider(paths, args.overlay_ports.get()); - // Note: action_plan will hold raw pointers to SourceControlFiles from this map + // Note: action_plan will hold raw pointers to SourceControlFileLocations from this map std::vector action_plan = create_feature_install_plan(provider, FullPackageSpec::to_feature_specs(specs), status_db); diff --git a/toolsrc/src/vcpkg/postbuildlint.cpp b/toolsrc/src/vcpkg/postbuildlint.cpp index d6581c2b4d..c760a034f3 100644 --- a/toolsrc/src/vcpkg/postbuildlint.cpp +++ b/toolsrc/src/vcpkg/postbuildlint.cpp @@ -857,14 +857,15 @@ namespace vcpkg::PostBuildLint size_t perform_all_checks(const PackageSpec& spec, const VcpkgPaths& paths, const PreBuildInfo& pre_build_info, - const BuildInfo& build_info) + const BuildInfo& build_info, + const fs::path& port_dir) { System::print2("-- Performing post-build validation\n"); const size_t error_count = perform_all_checks_and_return_error_count(spec, paths, pre_build_info, build_info); if (error_count != 0) { - const fs::path portfile = paths.ports / spec.name() / "portfile.cmake"; + const fs::path portfile = port_dir / "portfile.cmake"; System::print2(System::Color::error, "Found ", error_count, diff --git a/toolsrc/src/vcpkg/remove.cpp b/toolsrc/src/vcpkg/remove.cpp index f997667ace..a40b27bd74 100644 --- a/toolsrc/src/vcpkg/remove.cpp +++ b/toolsrc/src/vcpkg/remove.cpp @@ -229,7 +229,8 @@ namespace vcpkg::Remove Checks::exit_fail(VCPKG_LINE_INFO); } - Dependencies::PathsPortFileProvider provider(paths); + // Load ports from ports dirs + Dependencies::PathsPortFileProvider provider(paths, args.overlay_ports.get()); specs = Util::fmap(Update::find_outdated_packages(provider, status_db), [](auto&& outdated) { return outdated.spec; }); diff --git a/toolsrc/src/vcpkg/update.cpp b/toolsrc/src/vcpkg/update.cpp index 2c52d59524..6320bae5b7 100644 --- a/toolsrc/src/vcpkg/update.cpp +++ b/toolsrc/src/vcpkg/update.cpp @@ -23,10 +23,10 @@ namespace vcpkg::Update for (auto&& ipv : installed_packages) { const auto& pgh = ipv.core; - auto maybe_scf = provider.get_control_file(pgh->package.spec.name()); - if (auto p_scf = maybe_scf.get()) + auto maybe_scfl = provider.get_control_file(pgh->package.spec.name()); + if (auto p_scfl = maybe_scfl.get()) { - auto&& port_version = p_scf->core_paragraph->version; + auto&& port_version = p_scfl->source_control_file->core_paragraph->version; auto&& installed_version = pgh->package.version; if (installed_version != port_version) { @@ -57,7 +57,7 @@ namespace vcpkg::Update const StatusParagraphs status_db = database_load_check(paths); - Dependencies::PathsPortFileProvider provider(paths); + Dependencies::PathsPortFileProvider provider(paths, args.overlay_ports.get()); const auto outdated_packages = SortedVector(find_outdated_packages(provider, status_db), &OutdatedPackage::compare_by_name); diff --git a/toolsrc/src/vcpkg/vcpkgcmdarguments.cpp b/toolsrc/src/vcpkg/vcpkgcmdarguments.cpp index 8565c28f97..21bf4d0285 100644 --- a/toolsrc/src/vcpkg/vcpkgcmdarguments.cpp +++ b/toolsrc/src/vcpkg/vcpkgcmdarguments.cpp @@ -45,6 +45,45 @@ namespace vcpkg option_field = new_setting; } + static void parse_multivalue(const std::string* arg_begin, + const std::string* arg_end, + const std::string& option_name, + std::unique_ptr>& option_field) + { + if (arg_begin == arg_end) + { + System::print2(System::Color::error, "Error: expected value after ", option_name, '\n'); + Metrics::g_metrics.lock()->track_property("error", "error option name"); + Help::print_usage(); + Checks::exit_fail(VCPKG_LINE_INFO); + } + + if (!option_field) + { + option_field = std::make_unique>(); + } + option_field->emplace_back(*arg_begin); + } + + static void parse_cojoined_multivalue(std::string new_value, + const std::string& option_name, + std::unique_ptr>& option_field) + { + if (new_value.empty()) + { + System::print2(System::Color::error, "Error: expected value after ", option_name, '\n'); + Metrics::g_metrics.lock()->track_property("error", "error option name"); + Help::print_usage(); + Checks::exit_fail(VCPKG_LINE_INFO); + } + + if (!option_field) + { + option_field = std::make_unique>(); + } + option_field->emplace_back(std::move(new_value)); + } + VcpkgCmdArguments VcpkgCmdArguments::create_from_command_line(const int argc, const CommandLineCharType* const* const argv) { @@ -117,6 +156,13 @@ namespace vcpkg parse_value(arg_begin, arg_end, "--triplet", args.triplet); continue; } + if (Strings::starts_with(arg, "--overlay-ports=")) + { + parse_cojoined_multivalue(arg.substr(sizeof("--overlay-ports=") - 1), + "--overlay-ports", + args.overlay_ports); + continue; + } if (arg == "--debug") { parse_switch(true, "debug", args.debug); @@ -166,7 +212,21 @@ namespace vcpkg const auto eq_pos = arg.find('='); if (eq_pos != std::string::npos) { - args.optional_command_arguments.emplace(arg.substr(0, eq_pos), arg.substr(eq_pos + 1)); + const auto& key = arg.substr(0, eq_pos); + const auto& value = arg.substr(eq_pos + 1); + + auto it = args.optional_command_arguments.find(key); + if (args.optional_command_arguments.end() == it) + { + args.optional_command_arguments.emplace(key, std::vector { value }); + } + else + { + if (auto* maybe_values = it->second.get()) + { + maybe_values->emplace_back(value); + } + } } else { @@ -264,7 +324,51 @@ namespace vcpkg } else { - output.settings.emplace(option.name, it->second.value_or_exit(VCPKG_LINE_INFO)); + const auto& value = it->second.value_or_exit(VCPKG_LINE_INFO); + if (value.front().empty()) + { + // Fail when not given a value, e.g.: "vcpkg install sqlite3 --additional-ports=" + System::printf( + System::Color::error, "Error: The option '%s' must be passed an argument.\n", option.name); + failed = true; + } + else + { + output.settings.emplace(option.name, value.front()); + options_copy.erase(it); + } + } + } + } + + for (auto&& option : command_structure.options.multisettings) + { + const auto it = options_copy.find(option.name); + if (it != options_copy.end()) + { + if (!it->second.has_value()) + { + // Not having a string value indicates it was passed like '--a' + System::printf( + System::Color::error, "Error: The option '%s' must be passed an argument.\n", option.name); + failed = true; + } + else + { + const auto& value = it->second.value_or_exit(VCPKG_LINE_INFO); + for (auto&& v : value) + { + if (v.empty()) + { + System::printf( + System::Color::error, "Error: The option '%s' must be passed an argument.\n", option.name); + failed = true; + } + else + { + output.multisettings[option.name].emplace_back(v); + } + } options_copy.erase(it); } } @@ -306,7 +410,14 @@ namespace vcpkg { System::printf(" %-40s %s\n", (option.name + "=..."), option.short_help_text); } + for (auto&& option : command_structure.options.multisettings) + { + System::printf(" %-40s %s\n", (option.name + "=..."), option.short_help_text); + } System::printf(" %-40s %s\n", "--triplet ", "Set the default triplet for unqualified packages"); + System::printf(" %-40s %s\n", + "--overlay-ports=", + "Specify directories to be used when searching for ports"); System::printf(" %-40s %s\n", "--vcpkg-root ", "Specify the vcpkg directory to use instead of current directory or tool directory"); diff --git a/toolsrc/src/vcpkg/vcpkgpaths.cpp b/toolsrc/src/vcpkg/vcpkgpaths.cpp index 512bcfc35c..562a18c9ba 100644 --- a/toolsrc/src/vcpkg/vcpkgpaths.cpp +++ b/toolsrc/src/vcpkg/vcpkgpaths.cpp @@ -81,9 +81,6 @@ namespace vcpkg fs::path VcpkgPaths::package_dir(const PackageSpec& spec) const { return this->packages / spec.dir(); } - fs::path VcpkgPaths::port_dir(const PackageSpec& spec) const { return this->ports / spec.name(); } - fs::path VcpkgPaths::port_dir(const std::string& name) const { return this->ports / name; } - fs::path VcpkgPaths::build_info_file_path(const PackageSpec& spec) const { return this->package_dir(spec) / "BUILD_INFO"; diff --git a/toolsrc/vcpkg/vcpkg.vcxproj b/toolsrc/vcpkg/vcpkg.vcxproj index 8edea2244f..06d849a740 100644 --- a/toolsrc/vcpkg/vcpkg.vcxproj +++ b/toolsrc/vcpkg/vcpkg.vcxproj @@ -21,8 +21,8 @@ {34671B80-54F9-46F5-8310-AC429C11D4FB} vcpkg - 8.1 - v140 + 10.0 + v142 diff --git a/toolsrc/vcpkglib/vcpkglib.vcxproj b/toolsrc/vcpkglib/vcpkglib.vcxproj index a64eb42fd2..2a6853b8a9 100644 --- a/toolsrc/vcpkglib/vcpkglib.vcxproj +++ b/toolsrc/vcpkglib/vcpkglib.vcxproj @@ -21,8 +21,8 @@ {B98C92B7-2874-4537-9D46-D14E5C237F04} vcpkglib - 8.1 - v140 + 10.0 + v142 diff --git a/toolsrc/vcpkgmetricsuploader/vcpkgmetricsuploader.vcxproj b/toolsrc/vcpkgmetricsuploader/vcpkgmetricsuploader.vcxproj index e533d0e15d..2e689c7fca 100644 --- a/toolsrc/vcpkgmetricsuploader/vcpkgmetricsuploader.vcxproj +++ b/toolsrc/vcpkgmetricsuploader/vcpkgmetricsuploader.vcxproj @@ -21,8 +21,8 @@ {7D6FDEEB-B299-4A23-85EE-F67C4DED47BE} vcpkgmetricsuploader - 8.1 - v140 + 10.0 + v142 diff --git a/toolsrc/vcpkgtest/vcpkgtest.vcxproj b/toolsrc/vcpkgtest/vcpkgtest.vcxproj index 4cda294618..a7517ec541 100644 --- a/toolsrc/vcpkgtest/vcpkgtest.vcxproj +++ b/toolsrc/vcpkgtest/vcpkgtest.vcxproj @@ -48,8 +48,8 @@ {F27B8DB0-1279-4AF8-A2E3-1D49C4F0220D} Win32Proj vcpkgtest - 8.1 - v140 + 10.0 + v142 @@ -84,7 +84,7 @@ - + $(SolutionDir)msbuild.x86.debug\$(ProjectName)\ $(SolutionDir)msbuild.x86.debug\