Compare commits

..

31 Commits

Author SHA1 Message Date
Angela P Wen 17573ee1cc Merge pull request #1534 from github/update-v2.2.4-40babc141
Merge main into releases/v2
2023-02-10 10:20:44 -08:00
github-actions[bot] b6975b4b1a Update changelog for v2.2.4 2023-02-10 17:42:05 +00:00
Angela P Wen 40babc141f Tools telemetry: accurately report when feature flags were inaccessible (#1532)
* Cache whether feature flags are accessible

* Small comment fixup from linting change
2023-02-10 09:06:43 -08:00
Chuan-kai Lin 7ba5ed7eed Merge pull request #1531 from github/mergeback/v2.2.3-to-main-8775e868
Mergeback v2.2.3 refs/heads/releases/v2 into main
2023-02-08 13:06:40 -08:00
github-actions[bot] 21f3020df6 Update checked-in dependencies 2023-02-08 20:40:37 +00:00
github-actions[bot] b872c5adfd Update changelog and version after v2.2.3 2023-02-08 20:37:07 +00:00
Chuan-kai Lin 8775e86802 Merge pull request #1530 from github/update-v2.2.3-c4e22e9fc
Merge main into releases/v2
2023-02-08 12:35:06 -08:00
github-actions[bot] a2ad80b966 Update changelog for v2.2.3 2023-02-08 19:08:32 +00:00
Henry Mercer c4e22e9fce Merge pull request #1529 from github/henrymercer/remove-bypass-toolcache-flags
Remove feature flags for bypassing the toolcache
2023-02-08 18:13:01 +00:00
Henry Mercer db534af2ae Remove feature flags for bypassing the toolcache
- We can now use the default bundle version feature flags to remediate a
bad bundle update.
- Controlled switchover ensures that a repo consistently gets the same
bundle version, so we no longer have alert churn concerns with Kotlin
and Swift.
2023-02-08 15:20:51 +00:00
Chuan-kai Lin 4369dda4ae Merge pull request #1518 from github/cklin/codeql-cli-2.12.2
Bump default CodeQL version to 2.12.2
2023-02-07 10:27:54 -08:00
Chuan-kai Lin 4f08c2cf20 Bump default CodeQL version to 2.12.2 2023-02-07 08:10:01 -08:00
Angela P Wen 81644f35ff Add max line length of 120 to linter (#1524) 2023-02-07 14:09:33 +00:00
Henry Mercer 9ab6aa64a0 Merge pull request #1526 from github/mergeback/v2.2.2-to-main-39d8d7e7
Mergeback v2.2.2 refs/heads/releases/v2 into main
2023-02-06 20:23:48 +00:00
github-actions[bot] 256973e279 Update checked-in dependencies 2023-02-06 20:02:57 +00:00
github-actions[bot] 59b25b480f Update changelog and version after v2.2.2 2023-02-06 19:48:14 +00:00
Henry Mercer 39d8d7e78f Merge pull request #1525 from github/update-v2.2.2-927de483f
Merge main into releases/v2
2023-02-06 19:46:06 +00:00
Angela P Wen 39c954c513 Support security-experimental as a well-known suite (#1519) 2023-02-06 19:26:03 +00:00
github-actions[bot] 8af83634ca Update changelog for v2.2.2 2023-02-06 19:16:08 +00:00
Henry Mercer 927de483f0 Merge pull request #1523 from github/henrymercer/fix/cli-version-for-different-bundles
Fix toolcache behavior when downloading bundle from another repo
2023-02-06 19:05:45 +00:00
Henry Mercer e4c0a1b24d Merge branch 'main' into henrymercer/fix/cli-version-for-different-bundles 2023-02-06 18:24:11 +00:00
Henry Mercer d3962273b3 Merge pull request #1517 from github/henrymercer/fix/not-all-bundle-urls-contain-tag
Fix assumption that all CodeQL bundle URLs contain the tag name of the bundle
2023-02-06 18:20:21 +00:00
Henry Mercer c3cb270725 Merge pull request #1521 from MahmoudMabrok/patch-1
docs: add direct link to website
2023-02-06 16:34:01 +00:00
Henry Mercer 2b674f7ab9 Fix toolcache behavior when downloading bundle from another repo 2023-02-06 16:25:07 +00:00
Henry Mercer 6d47a7c8b1 Add regression test for bundle from different repo 2023-02-06 16:25:07 +00:00
Henry Mercer c6ff11c1c4 Add changelog note 2023-02-06 16:24:25 +00:00
Henry Mercer d3f2b2e6d2 Warn when multiple bundles for a single CLI are found in the toolcache 2023-02-06 12:28:33 +00:00
Henry Mercer d49282c3b5 Rename forceLatest to forceShippedTools 2023-02-06 11:57:48 +00:00
Mahmoud Mabrok Fouad c5c475188a docs: add direct link to website
To make it easy for users to go to website for more info.
2023-02-05 13:56:35 +02:00
Henry Mercer f140af5e28 Refactor setting up CodeQL to handle bundle URLs without tags 2023-02-03 19:15:06 +00:00
Henry Mercer e0fc1c91b2 Add regression test for a bundle URL without a tag 2023-02-03 19:13:24 +00:00
44 changed files with 709 additions and 808 deletions
+6
View File
@@ -33,6 +33,12 @@
"alphabetize": {"order": "asc"},
"newlines-between": "always"
}],
"max-len": ["error", {
"code": 120,
"ignoreUrls": true,
"ignoreStrings": true,
"ignoreTemplateLiterals": true
}],
"no-async-foreach/no-async-foreach": "error",
"no-console": "off",
"no-sequences": "error",
+9 -1
View File
@@ -1,9 +1,17 @@
# CodeQL Action Changelog
## [UNRELEASED]
## 2.2.4 - 10 Feb 2023
No user facing changes.
## 2.2.3 - 08 Feb 2023
- Update default CodeQL bundle version to 2.12.2. [#1518](https://github.com/github/codeql-action/pull/1518)
## 2.2.2 - 06 Feb 2023
- Fix an issue where customers using the CodeQL Action with the [CodeQL Action sync tool](https://docs.github.com/en/enterprise-server@3.7/admin/code-security/managing-github-advanced-security-for-your-enterprise/configuring-code-scanning-for-your-appliance#configuring-codeql-analysis-on-a-server-without-internet-access) would not be able to obtain the CodeQL tools. [#1517](https://github.com/github/codeql-action/pull/1517)
## 2.2.1 - 27 Jan 2023
No user facing changes.
+1 -1
View File
@@ -1,6 +1,6 @@
# CodeQL Action
This action runs GitHub's industry-leading semantic code analysis engine, CodeQL, against a repository's source code to find security vulnerabilities. It then automatically uploads the results to GitHub so they can be displayed in the repository's security tab. CodeQL runs an extensible set of [queries](https://github.com/github/codeql), which have been developed by the community and the [GitHub Security Lab](https://securitylab.github.com/) to find common vulnerabilities in your code.
This action runs GitHub's industry-leading semantic code analysis engine, [CodeQL](https://codeql.github.com/), against a repository's source code to find security vulnerabilities. It then automatically uploads the results to GitHub so they can be displayed in the repository's security tab. CodeQL runs an extensible set of [queries](https://github.com/github/codeql), which have been developed by the community and the [GitHub Security Lab](https://securitylab.github.com/) to find common vulnerabilities in your code.
For a list of recent changes, see the CodeQL Action's [changelog](CHANGELOG.md).
Generated
+7 -4
View File
@@ -23,7 +23,7 @@ var __importStar = (this && this.__importStar) || function (mod) {
return result;
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.getExtraOptions = exports.getCodeQLForCmd = exports.getCodeQLForTesting = exports.getCachedCodeQL = exports.setCodeQL = exports.getCodeQL = exports.setupCodeQL = exports.CODEQL_VERSION_BETTER_RESOLVE_LANGUAGES = exports.CODEQL_VERSION_ML_POWERED_QUERIES_WINDOWS = exports.CODEQL_VERSION_TRACING_GLIBC_2_34 = exports.CODEQL_VERSION_NEW_TRACING = exports.CODEQL_VERSION_GHES_PACK_DOWNLOAD = exports.CommandInvocationError = void 0;
exports.getExtraOptions = exports.getCodeQLForCmd = exports.getCodeQLForTesting = exports.getCachedCodeQL = exports.setCodeQL = exports.getCodeQL = exports.setupCodeQL = exports.CODEQL_VERSION_SECURITY_EXPERIMENTAL_SUITE = exports.CODEQL_VERSION_BETTER_RESOLVE_LANGUAGES = exports.CODEQL_VERSION_ML_POWERED_QUERIES_WINDOWS = exports.CODEQL_VERSION_TRACING_GLIBC_2_34 = exports.CODEQL_VERSION_NEW_TRACING = exports.CODEQL_VERSION_GHES_PACK_DOWNLOAD = exports.CommandInvocationError = void 0;
const fs = __importStar(require("fs"));
const path = __importStar(require("path"));
const toolrunner = __importStar(require("@actions/exec/lib/toolrunner"));
@@ -94,6 +94,10 @@ exports.CODEQL_VERSION_ML_POWERED_QUERIES_WINDOWS = "2.9.0";
* --extractor-options-verbosity that we need.
*/
exports.CODEQL_VERSION_BETTER_RESOLVE_LANGUAGES = "2.10.3";
/**
* Versions 2.11.1+ of the CodeQL Bundle include a `security-experimental` built-in query suite for each language.
*/
exports.CODEQL_VERSION_SECURITY_EXPERIMENTAL_SUITE = "2.12.1";
/**
* Set up CodeQL CLI access.
*
@@ -101,16 +105,15 @@ exports.CODEQL_VERSION_BETTER_RESOLVE_LANGUAGES = "2.10.3";
* @param apiDetails
* @param tempDir
* @param variant
* @param bypassToolcache
* @param defaultCliVersion
* @param logger
* @param checkVersion Whether to check that CodeQL CLI meets the minimum
* version requirement. Must be set to true outside tests.
* @returns a { CodeQL, toolsVersion } object.
*/
async function setupCodeQL(toolsInput, apiDetails, tempDir, variant, bypassToolcache, defaultCliVersion, logger, checkVersion) {
async function setupCodeQL(toolsInput, apiDetails, tempDir, variant, defaultCliVersion, logger, checkVersion) {
try {
const { codeqlFolder, toolsDownloadDurationMs, toolsSource, toolsVersion } = await setupCodeql.setupCodeQLBundle(toolsInput, apiDetails, tempDir, variant, bypassToolcache, defaultCliVersion, logger);
const { codeqlFolder, toolsDownloadDurationMs, toolsSource, toolsVersion } = await setupCodeql.setupCodeQLBundle(toolsInput, apiDetails, tempDir, variant, defaultCliVersion, logger);
let codeqlCmd = path.join(codeqlFolder, "codeql", "codeql");
if (process.platform === "win32") {
codeqlCmd += ".exe";
+1 -1
View File
File diff suppressed because one or more lines are too long
+70 -38
View File
@@ -97,7 +97,7 @@ ava_1.default.beforeEach(() => {
* @returns the download URL for the bundle. This can be passed to the tools parameter of
* `codeql.setupCodeQL`.
*/
function mockDownloadApi({ apiDetails = sampleApiDetails, isPinned, tagName, }) {
function mockDownloadApi({ apiDetails = sampleApiDetails, isPinned, repo = "github/codeql-action", platformSpecific = true, tagName, }) {
const platform = process.platform === "win32"
? "win64"
: process.platform === "linux"
@@ -105,7 +105,7 @@ function mockDownloadApi({ apiDetails = sampleApiDetails, isPinned, tagName, })
: "osx64";
const baseUrl = apiDetails?.url ?? "https://example.com";
const relativeUrl = apiDetails
? `/github/codeql-action/releases/download/${tagName}/codeql-bundle-${platform}.tar.gz`
? `/${repo}/releases/download/${tagName}/codeql-bundle${platformSpecific ? `-${platform}` : ""}.tar.gz`
: `/download/${tagName}/codeql-bundle.tar.gz`;
(0, nock_1.default)(baseUrl)
.get(relativeUrl)
@@ -114,7 +114,7 @@ function mockDownloadApi({ apiDetails = sampleApiDetails, isPinned, tagName, })
}
async function installIntoToolcache({ apiDetails = sampleApiDetails, cliVersion, isPinned, tagName, tmpDir, }) {
const url = mockDownloadApi({ apiDetails, isPinned, tagName });
await codeql.setupCodeQL(cliVersion !== undefined ? undefined : url, apiDetails, tmpDir, util.GitHubVariant.GHES, false, cliVersion !== undefined
await codeql.setupCodeQL(cliVersion !== undefined ? undefined : url, apiDetails, tmpDir, util.GitHubVariant.GHES, cliVersion !== undefined
? { cliVersion, tagName, variant: util.GitHubVariant.GHES }
: SAMPLE_DEFAULT_CLI_VERSION, (0, logging_1.getRunnerLogger)(true), false);
}
@@ -153,7 +153,7 @@ function mockApiDetails(apiDetails) {
tagName: `codeql-bundle-${version}`,
isPinned: false,
});
const result = await codeql.setupCodeQL(url, sampleApiDetails, tmpDir, util.GitHubVariant.DOTCOM, false, SAMPLE_DEFAULT_CLI_VERSION, (0, logging_1.getRunnerLogger)(true), false);
const result = await codeql.setupCodeQL(url, sampleApiDetails, tmpDir, util.GitHubVariant.DOTCOM, SAMPLE_DEFAULT_CLI_VERSION, (0, logging_1.getRunnerLogger)(true), false);
t.assert(toolcache.find("CodeQL", `0.0.0-${version}`));
t.is(result.toolsVersion, `0.0.0-${version}`);
t.is(result.toolsSource, init_1.ToolsSource.Download);
@@ -173,7 +173,7 @@ function mockApiDetails(apiDetails) {
const url = mockDownloadApi({
tagName: "codeql-bundle-20200610",
});
const result = await codeql.setupCodeQL(url, sampleApiDetails, tmpDir, util.GitHubVariant.DOTCOM, false, SAMPLE_DEFAULT_CLI_VERSION, (0, logging_1.getRunnerLogger)(true), false);
const result = await codeql.setupCodeQL(url, sampleApiDetails, tmpDir, util.GitHubVariant.DOTCOM, SAMPLE_DEFAULT_CLI_VERSION, (0, logging_1.getRunnerLogger)(true), false);
t.assert(toolcache.find("CodeQL", "0.0.0-20200610"));
t.deepEqual(result.toolsVersion, "0.0.0-20200610");
t.is(result.toolsSource, init_1.ToolsSource.Download);
@@ -207,7 +207,7 @@ for (const { cliVersion, expectedToolcacheVersion, } of EXPLICITLY_REQUESTED_BUN
const url = mockDownloadApi({
tagName: "codeql-bundle-20200610",
});
const result = await codeql.setupCodeQL(url, sampleApiDetails, tmpDir, util.GitHubVariant.DOTCOM, false, SAMPLE_DEFAULT_CLI_VERSION, (0, logging_1.getRunnerLogger)(true), false);
const result = await codeql.setupCodeQL(url, sampleApiDetails, tmpDir, util.GitHubVariant.DOTCOM, SAMPLE_DEFAULT_CLI_VERSION, (0, logging_1.getRunnerLogger)(true), false);
t.assert(releaseApiMock.isDone(), "Releases API should have been called");
t.assert(toolcache.find("CodeQL", expectedToolcacheVersion));
t.deepEqual(result.toolsVersion, cliVersion);
@@ -256,7 +256,7 @@ for (const { githubReleases, toolcacheVersion } of [
}))),
}));
}
const result = await codeql.setupCodeQL(undefined, sampleApiDetails, tmpDir, util.GitHubVariant.DOTCOM, false, SAMPLE_DEFAULT_CLI_VERSION, (0, logging_1.getRunnerLogger)(true), false);
const result = await codeql.setupCodeQL(undefined, sampleApiDetails, tmpDir, util.GitHubVariant.DOTCOM, SAMPLE_DEFAULT_CLI_VERSION, (0, logging_1.getRunnerLogger)(true), false);
t.is(result.toolsVersion, SAMPLE_DEFAULT_CLI_VERSION.cliVersion);
t.is(result.toolsSource, init_1.ToolsSource.Toolcache);
t.is(result.toolsDownloadDurationMs, undefined);
@@ -272,7 +272,7 @@ for (const variant of [util.GitHubVariant.GHAE, util.GitHubVariant.GHES]) {
isPinned: true,
tmpDir,
});
const result = await codeql.setupCodeQL(undefined, sampleApiDetails, tmpDir, variant, false, {
const result = await codeql.setupCodeQL(undefined, sampleApiDetails, tmpDir, variant, {
cliVersion: defaults.cliVersion,
tagName: defaults.bundleVersion,
variant,
@@ -295,7 +295,7 @@ for (const variant of [util.GitHubVariant.GHAE, util.GitHubVariant.GHES]) {
mockDownloadApi({
tagName: defaults.bundleVersion,
});
const result = await codeql.setupCodeQL(undefined, sampleApiDetails, tmpDir, variant, false, {
const result = await codeql.setupCodeQL(undefined, sampleApiDetails, tmpDir, variant, {
cliVersion: defaults.cliVersion,
tagName: defaults.bundleVersion,
variant,
@@ -319,7 +319,7 @@ for (const variant of [util.GitHubVariant.GHAE, util.GitHubVariant.GHES]) {
mockDownloadApi({
tagName: defaults.bundleVersion,
});
const result = await codeql.setupCodeQL("latest", sampleApiDetails, tmpDir, util.GitHubVariant.DOTCOM, false, SAMPLE_DEFAULT_CLI_VERSION, (0, logging_1.getRunnerLogger)(true), false);
const result = await codeql.setupCodeQL("latest", sampleApiDetails, tmpDir, util.GitHubVariant.DOTCOM, SAMPLE_DEFAULT_CLI_VERSION, (0, logging_1.getRunnerLogger)(true), false);
t.deepEqual(result.toolsVersion, defaults.cliVersion);
t.is(result.toolsSource, init_1.ToolsSource.Download);
t.assert(Number.isInteger(result.toolsDownloadDurationMs));
@@ -327,41 +327,73 @@ for (const variant of [util.GitHubVariant.GHAE, util.GitHubVariant.GHES]) {
t.is(cachedVersions.length, 2);
});
});
(0, ava_1.default)("download codeql bundle from github ae endpoint", async (t) => {
for (const isBundleVersionInUrl of [true, false]) {
const inclusionString = isBundleVersionInUrl
? "includes"
: "does not include";
(0, ava_1.default)(`download codeql bundle from github ae endpoint (URL ${inclusionString} bundle version)`, async (t) => {
await util.withTmpDir(async (tmpDir) => {
(0, testing_utils_1.setupActionsVars)(tmpDir, tmpDir);
const bundleAssetID = 10;
const platform = process.platform === "win32"
? "win64"
: process.platform === "linux"
? "linux64"
: "osx64";
const codeQLBundleName = `codeql-bundle-${platform}.tar.gz`;
const eventualDownloadUrl = isBundleVersionInUrl
? `https://example.githubenterprise.com/github/codeql-action/releases/download/${defaults.bundleVersion}/${codeQLBundleName}`
: `https://example.githubenterprise.com/api/v3/repos/github/codeql-action/releases/assets/${bundleAssetID}`;
(0, nock_1.default)("https://example.githubenterprise.com")
.get(`/api/v3/enterprise/code-scanning/codeql-bundle/find/${defaults.bundleVersion}`)
.reply(200, {
assets: { [codeQLBundleName]: bundleAssetID },
});
(0, nock_1.default)("https://example.githubenterprise.com")
.get(`/api/v3/enterprise/code-scanning/codeql-bundle/download/${bundleAssetID}`)
.reply(200, {
url: eventualDownloadUrl,
});
(0, nock_1.default)("https://example.githubenterprise.com")
.get(eventualDownloadUrl.replace("https://example.githubenterprise.com", ""))
.replyWithFile(200, path_1.default.join(__dirname, `/../src/testdata/codeql-bundle-pinned.tar.gz`));
mockApiDetails(sampleGHAEApiDetails);
sinon.stub(actionsUtil, "isRunningLocalAction").returns(false);
process.env["GITHUB_ACTION_REPOSITORY"] = "github/codeql-action";
const result = await codeql.setupCodeQL(undefined, sampleGHAEApiDetails, tmpDir, util.GitHubVariant.GHAE, {
cliVersion: defaults.cliVersion,
tagName: defaults.bundleVersion,
variant: util.GitHubVariant.GHAE,
}, (0, logging_1.getRunnerLogger)(true), false);
t.is(result.toolsSource, init_1.ToolsSource.Download);
t.assert(Number.isInteger(result.toolsDownloadDurationMs));
const cachedVersions = toolcache.findAllVersions("CodeQL");
t.is(cachedVersions.length, 1);
});
});
}
(0, ava_1.default)("bundle URL from another repo is cached as 0.0.0-bundleVersion", async (t) => {
await util.withTmpDir(async (tmpDir) => {
(0, testing_utils_1.setupActionsVars)(tmpDir, tmpDir);
const bundleAssetID = 10;
const platform = process.platform === "win32"
? "win64"
: process.platform === "linux"
? "linux64"
: "osx64";
const codeQLBundleName = `codeql-bundle-${platform}.tar.gz`;
(0, nock_1.default)("https://example.githubenterprise.com")
.get(`/api/v3/enterprise/code-scanning/codeql-bundle/find/${defaults.bundleVersion}`)
.reply(200, {
assets: { [codeQLBundleName]: bundleAssetID },
mockApiDetails(sampleApiDetails);
sinon.stub(actionsUtil, "isRunningLocalAction").returns(true);
const releasesApiMock = mockReleaseApi({
assetNames: ["cli-version-2.12.2.txt"],
tagName: "codeql-bundle-20230203",
});
(0, nock_1.default)("https://example.githubenterprise.com")
.get(`/api/v3/enterprise/code-scanning/codeql-bundle/download/${bundleAssetID}`)
.reply(200, {
url: `https://example.githubenterprise.com/github/codeql-action/releases/download/${defaults.bundleVersion}/${codeQLBundleName}`,
mockDownloadApi({
repo: "dsp-testing/codeql-cli-nightlies",
platformSpecific: false,
tagName: "codeql-bundle-20230203",
});
(0, nock_1.default)("https://example.githubenterprise.com")
.get(`/github/codeql-action/releases/download/${defaults.bundleVersion}/${codeQLBundleName}`)
.replyWithFile(200, path_1.default.join(__dirname, `/../src/testdata/codeql-bundle-pinned.tar.gz`));
mockApiDetails(sampleGHAEApiDetails);
sinon.stub(actionsUtil, "isRunningLocalAction").returns(false);
process.env["GITHUB_ACTION_REPOSITORY"] = "github/codeql-action";
const result = await codeql.setupCodeQL(undefined, sampleGHAEApiDetails, tmpDir, util.GitHubVariant.GHAE, false, {
cliVersion: defaults.cliVersion,
tagName: defaults.bundleVersion,
variant: util.GitHubVariant.GHAE,
}, (0, logging_1.getRunnerLogger)(true), false);
const result = await codeql.setupCodeQL("https://github.com/dsp-testing/codeql-cli-nightlies/releases/download/codeql-bundle-20230203/codeql-bundle.tar.gz", sampleApiDetails, tmpDir, util.GitHubVariant.DOTCOM, SAMPLE_DEFAULT_CLI_VERSION, (0, logging_1.getRunnerLogger)(true), false);
t.is(result.toolsVersion, "0.0.0-20230203");
t.is(result.toolsSource, init_1.ToolsSource.Download);
t.assert(Number.isInteger(result.toolsDownloadDurationMs));
t.true(Number.isInteger(result.toolsDownloadDurationMs));
const cachedVersions = toolcache.findAllVersions("CodeQL");
t.is(cachedVersions.length, 1);
t.is(cachedVersions[0], "0.0.0-20230203");
t.false(releasesApiMock.isDone());
});
});
(0, ava_1.default)("getExtraOptions works for explicit paths", (t) => {
File diff suppressed because one or more lines are too long
+16 -3
View File
@@ -131,7 +131,11 @@ async function addDefaultQueries(codeQL, languages, resultMap) {
await runResolveQueries(codeQL, resultMap, suites, undefined);
}
// The set of acceptable values for built-in suites from the codeql bundle
const builtinSuites = ["security-extended", "security-and-quality"];
const builtinSuites = [
"security-experimental",
"security-extended",
"security-and-quality",
];
/**
* Determine the set of queries associated with suiteName's suites and add them to resultMap.
* Throws an error if suiteName is not a valid builtin suite.
@@ -143,6 +147,12 @@ async function addBuiltinSuiteQueries(languages, codeQL, resultMap, packs, suite
if (!found) {
throw new Error(getQueryUsesInvalid(configFile, suiteName));
}
if (suiteName === "security-experimental" &&
!(await (0, util_1.codeQlVersionAbove)(codeQL, codeql_1.CODEQL_VERSION_SECURITY_EXPERIMENTAL_SUITE))) {
throw new Error(`The 'security-experimental' suite is not supported on CodeQL CLI versions earlier than
${codeql_1.CODEQL_VERSION_SECURITY_EXPERIMENTAL_SUITE}. Please upgrade to CodeQL CLI version
${codeql_1.CODEQL_VERSION_SECURITY_EXPERIMENTAL_SUITE} or later.`);
}
// If we're running the JavaScript security-extended analysis (or a superset of it), the repo is
// opted into the ML-powered queries beta, and a user hasn't already added the ML-powered query
// pack, then add the ML-powered query pack so that we run ML-powered queries.
@@ -151,7 +161,9 @@ async function addBuiltinSuiteQueries(languages, codeQL, resultMap, packs, suite
(process.platform !== "win32" ||
(await (0, util_1.codeQlVersionAbove)(codeQL, codeql_1.CODEQL_VERSION_ML_POWERED_QUERIES_WINDOWS))) &&
languages.includes("javascript") &&
(found === "security-extended" || found === "security-and-quality") &&
(found === "security-experimental" ||
found === "security-extended" ||
found === "security-and-quality") &&
!packs.javascript?.some(isMlPoweredJsQueriesPack) &&
(await featureEnablement.getValue(feature_flags_1.Feature.MlPoweredQueriesEnabled, codeQL))) {
if (!packs.javascript) {
@@ -893,7 +905,8 @@ exports.parsePacks = parsePacks;
* Without a '+', an input value will override the corresponding value in the config file.
*
* @param inputValue The input value to process.
* @returns true if the input value should replace the corresponding value in the config file, false if it should be appended.
* @returns true if the input value should replace the corresponding value in the config file,
* false if it should be appended.
*/
function shouldCombine(inputValue) {
return !!inputValue?.trim().startsWith("+");
File diff suppressed because one or more lines are too long
+4 -1
View File
@@ -1014,7 +1014,7 @@ const mlPoweredQueriesMacro = ava_1.default.macro({
// Test that the ~0.1.0 version of ML-powered queries is run on v2.8.3 of the CLI.
(0, ava_1.default)(mlPoweredQueriesMacro, "2.8.3", true, undefined, "security-extended", process.platform === "win32" ? undefined : "~0.1.0");
// Test that ML-powered queries aren't run when the user hasn't specified that we should run the
// `security-extended` or `security-and-quality` query suite.
// `security-extended`, `security-and-quality`, or `security-experimental` query suite.
(0, ava_1.default)(mlPoweredQueriesMacro, "2.7.5", true, undefined, undefined, undefined);
// Test that ML-powered queries are run on non-Windows platforms running `security-extended` on
// versions of the CodeQL CLI prior to 2.9.0.
@@ -1042,6 +1042,9 @@ const mlPoweredQueriesMacro = ava_1.default.macro({
// Test that ML-powered queries are run on all platforms running `security-and-quality` on CodeQL
// CLI 2.11.3+.
(0, ava_1.default)(mlPoweredQueriesMacro, "2.11.3", true, undefined, "security-and-quality", "~0.4.0");
// Test that ML-powered queries are run on all platforms running `security-experimental` on CodeQL
// CLI 2.12.1+.
(0, ava_1.default)(mlPoweredQueriesMacro, "2.12.1", true, undefined, "security-experimental", "~0.4.0");
const calculateAugmentationMacro = ava_1.default.macro({
exec: async (t, _title, rawPacksInput, rawQueriesInput, languages, expectedAugmentationProperties) => {
const actualAugmentationProperties = configUtils.calculateAugmentation(rawPacksInput, rawQueriesInput, languages);
File diff suppressed because one or more lines are too long
+4 -4
View File
@@ -1,6 +1,6 @@
{
"bundleVersion": "codeql-bundle-20230120",
"cliVersion": "2.12.1",
"priorBundleVersion": "codeql-bundle-20230105",
"priorCliVersion": "2.12.0"
"bundleVersion": "codeql-bundle-20230207",
"cliVersion": "2.12.2",
"priorBundleVersion": "codeql-bundle-20230120",
"priorCliVersion": "2.12.1"
}
+10 -22
View File
@@ -34,8 +34,6 @@ const DEFAULT_VERSION_FEATURE_FLAG_PREFIX = "default_codeql_version_";
const DEFAULT_VERSION_FEATURE_FLAG_SUFFIX = "_enabled";
var Feature;
(function (Feature) {
Feature["BypassToolcacheEnabled"] = "bypass_toolcache_enabled";
Feature["BypassToolcacheKotlinSwiftEnabled"] = "bypass_toolcache_kotlin_swift_enabled";
Feature["CliConfigFileEnabled"] = "cli_config_file_enabled";
Feature["DisableKotlinAnalysisEnabled"] = "disable_kotlin_analysis_enabled";
Feature["MlPoweredQueriesEnabled"] = "ml_powered_queries_enabled";
@@ -43,18 +41,6 @@ var Feature;
Feature["UploadFailedSarifEnabled"] = "upload_failed_sarif_enabled";
})(Feature = exports.Feature || (exports.Feature = {}));
exports.featureConfig = {
[Feature.BypassToolcacheEnabled]: {
envVar: "CODEQL_BYPASS_TOOLCACHE",
// Cannot specify a minimum version because this flag is checked before we have
// access to the CodeQL instance.
minimumVersion: undefined,
},
[Feature.BypassToolcacheKotlinSwiftEnabled]: {
envVar: "CODEQL_BYPASS_TOOLCACHE_KOTLIN_SWIFT",
// Cannot specify a minimum version because this flag is checked before we have
// access to the CodeQL instance.
minimumVersion: undefined,
},
[Feature.DisableKotlinAnalysisEnabled]: {
envVar: "CODEQL_DISABLE_KOTLIN_ANALYSIS",
minimumVersion: undefined,
@@ -105,10 +91,6 @@ class Features {
if (!codeql && exports.featureConfig[feature].minimumVersion) {
throw new Error(`Internal error: A minimum version is specified for feature ${feature}, but no instance of CodeQL was provided.`);
}
// Bypassing the toolcache is disabled in test mode.
if (feature === Feature.BypassToolcacheEnabled && util.isInTestMode()) {
return false;
}
const envVar = (process.env[exports.featureConfig[feature].envVar] || "").toLocaleLowerCase();
// Do not use this feature if user explicitly disables it via an environment variable.
if (envVar === "false") {
@@ -136,7 +118,7 @@ class GitHubFeatureFlags {
this.repositoryNwo = repositoryNwo;
this.featureFlagsFile = featureFlagsFile;
this.logger = logger;
/**/
this.hasAccessedRemoteFeatureFlags = false; // Not accessed by default.
}
getCliVersionFromFeatureFlag(f) {
if (!f.startsWith(DEFAULT_VERSION_FEATURE_FLAG_PREFIX) ||
@@ -157,7 +139,9 @@ class GitHubFeatureFlags {
const defaultDotComCliVersion = await this.getDefaultDotcomCliVersion();
return {
cliVersion: defaultDotComCliVersion.version,
toolsFeatureFlagsValid: defaultDotComCliVersion.toolsFeatureFlagsValid,
toolsFeatureFlagsValid: this.hasAccessedRemoteFeatureFlags
? defaultDotComCliVersion.toolsFeatureFlagsValid
: undefined,
variant,
};
}
@@ -188,7 +172,9 @@ class GitHubFeatureFlags {
`shipped with the Action. This is ${defaults.cliVersion}.`);
return {
version: defaults.cliVersion,
toolsFeatureFlagsValid: false,
toolsFeatureFlagsValid: this.hasAccessedRemoteFeatureFlags
? false
: undefined,
};
}
const maxCliVersion = enabledFeatureFlagCliVersions.reduce((maxVersion, currentVersion) => currentVersion > maxVersion ? currentVersion : maxVersion, enabledFeatureFlagCliVersions[0]);
@@ -255,6 +241,7 @@ class GitHubFeatureFlags {
// Do nothing when not running against github.com
if (this.gitHubVersion.type !== util.GitHubVariant.DOTCOM) {
this.logger.debug("Not running against github.com. Disabling all toggleable features.");
this.hasAccessedRemoteFeatureFlags = false;
return {};
}
try {
@@ -265,6 +252,7 @@ class GitHubFeatureFlags {
const remoteFlags = response.data;
this.logger.debug("Loaded the following default values for the feature flags from the Code Scanning API: " +
`${JSON.stringify(remoteFlags)}`);
this.hasAccessedRemoteFeatureFlags = true;
return remoteFlags;
}
catch (e) {
@@ -273,6 +261,7 @@ class GitHubFeatureFlags {
"As a result, it will not be opted into any experimental features. " +
"This could be because the Action is running on a pull request from a fork. If not, " +
`please ensure the Action has the 'security-events: write' permission. Details: ${e}`);
this.hasAccessedRemoteFeatureFlags = false;
return {};
}
else {
@@ -283,7 +272,6 @@ class GitHubFeatureFlags {
throw new Error(`Encountered an error while trying to determine feature enablement: ${e}`);
}
}
return {};
}
}
//# sourceMappingURL=feature-flags.js.map
File diff suppressed because one or more lines are too long
+1 -1
View File
@@ -122,7 +122,7 @@ async function run() {
if (codeQLDefaultVersionInfo.variant === util_1.GitHubVariant.DOTCOM) {
toolsFeatureFlagsValid = codeQLDefaultVersionInfo.toolsFeatureFlagsValid;
}
const initCodeQLResult = await (0, init_1.initCodeQL)((0, actions_util_1.getOptionalInput)("tools"), apiDetails, (0, actions_util_1.getTemporaryDirectory)(), gitHubVersion.type, await (0, util_1.shouldBypassToolcache)(features, (0, actions_util_1.getOptionalInput)("tools"), (0, actions_util_1.getOptionalInput)("languages"), repositoryNwo, logger), codeQLDefaultVersionInfo, logger);
const initCodeQLResult = await (0, init_1.initCodeQL)((0, actions_util_1.getOptionalInput)("tools"), apiDetails, (0, actions_util_1.getTemporaryDirectory)(), gitHubVersion.type, codeQLDefaultVersionInfo, logger);
codeql = initCodeQLResult.codeql;
toolsDownloadDurationMs = initCodeQLResult.toolsDownloadDurationMs;
toolsVersion = initCodeQLResult.toolsVersion;
File diff suppressed because one or more lines are too long
Generated
+2 -2
View File
@@ -41,9 +41,9 @@ var ToolsSource;
ToolsSource["Toolcache"] = "TOOLCACHE";
ToolsSource["Download"] = "DOWNLOAD";
})(ToolsSource = exports.ToolsSource || (exports.ToolsSource = {}));
async function initCodeQL(toolsInput, apiDetails, tempDir, variant, bypassToolcache, defaultCliVersion, logger) {
async function initCodeQL(toolsInput, apiDetails, tempDir, variant, defaultCliVersion, logger) {
logger.startGroup("Setup CodeQL tools");
const { codeql, toolsDownloadDurationMs, toolsSource, toolsVersion } = await (0, codeql_1.setupCodeQL)(toolsInput, apiDetails, tempDir, variant, bypassToolcache, defaultCliVersion, logger, true);
const { codeql, toolsDownloadDurationMs, toolsSource, toolsVersion } = await (0, codeql_1.setupCodeQL)(toolsInput, apiDetails, tempDir, variant, defaultCliVersion, logger, true);
await codeql.printVersion();
logger.endGroup();
return { codeql, toolsDownloadDurationMs, toolsSource, toolsVersion };
+1 -1
View File
@@ -1 +1 @@
{"version":3,"file":"init.js","sourceRoot":"","sources":["../src/init.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;;;;;;;;;AAAA,uCAAyB;AACzB,2CAA6B;AAE7B,yEAA2D;AAC3D,kEAAoD;AAEpD,gEAAkD;AAElD,qCAA2E;AAC3E,4DAA8C;AAI9C,mDAAwE;AACxE,6CAA+B;AAC/B,iCAA4C;AAE5C,IAAY,WAKX;AALD,WAAY,WAAW;IACrB,kCAAmB,CAAA;IACnB,8BAAe,CAAA;IACf,sCAAuB,CAAA;IACvB,oCAAqB,CAAA;AACvB,CAAC,EALW,WAAW,GAAX,mBAAW,KAAX,mBAAW,QAKtB;AAEM,KAAK,UAAU,UAAU,CAC9B,UAA8B,EAC9B,UAA4B,EAC5B,OAAe,EACf,OAA2B,EAC3B,eAAwB,EACxB,iBAA2C,EAC3C,MAAc;IAOd,MAAM,CAAC,UAAU,CAAC,oBAAoB,CAAC,CAAC;IACxC,MAAM,EAAE,MAAM,EAAE,uBAAuB,EAAE,WAAW,EAAE,YAAY,EAAE,GAClE,MAAM,IAAA,oBAAW,EACf,UAAU,EACV,UAAU,EACV,OAAO,EACP,OAAO,EACP,eAAe,EACf,iBAAiB,EACjB,MAAM,EACN,IAAI,CACL,CAAC;IACJ,MAAM,MAAM,CAAC,YAAY,EAAE,CAAC;IAC5B,MAAM,CAAC,QAAQ,EAAE,CAAC;IAClB,OAAO,EAAE,MAAM,EAAE,uBAAuB,EAAE,WAAW,EAAE,YAAY,EAAE,CAAC;AACxE,CAAC;AA7BD,gCA6BC;AAEM,KAAK,UAAU,UAAU,CAC9B,cAAkC,EAClC,YAAgC,EAChC,UAA8B,EAC9B,eAAmC,EACnC,UAA8B,EAC9B,UAA8B,EAC9B,kBAA2B,EAC3B,SAAkB,EAClB,iBAAyB,EACzB,iBAAyB,EACzB,UAAyB,EACzB,OAAe,EACf,MAAc,EACd,aAAqB,EACrB,aAAiC,EACjC,UAAoC,EACpC,iBAAoC,EACpC,MAAc;IAEd,MAAM,CAAC,UAAU,CAAC,6BAA6B,CAAC,CAAC;IACjD,MAAM,MAAM,GAAG,MAAM,WAAW,CAAC,UAAU,CACzC,cAAc,EACd,YAAY,EACZ,UAAU,EACV,eAAe,EACf,UAAU,EACV,UAAU,EACV,kBAAkB,EAClB,SAAS,EACT,iBAAiB,EACjB,iBAAiB,EACjB,UAAU,EACV,OAAO,EACP,MAAM,EACN,aAAa,EACb,aAAa,EACb,UAAU,EACV,iBAAiB,EACjB,MAAM,CACP,CAAC;IACF,aAAa,CAAC,uBAAuB,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IACtD,MAAM,CAAC,QAAQ,EAAE,CAAC;IAClB,OAAO,MAAM,CAAC;AAChB,CAAC;AA5CD,gCA4CC;AAEM,KAAK,UAAU,OAAO,CAC3B,MAAc,EACd,MAA0B,EAC1B,UAAkB,EAClB,WAA+B,EAC/B,iBAAoC,EACpC,MAAc;IAEd,EAAE,CAAC,SAAS,CAAC,MAAM,CAAC,UAAU,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;IAErD,IAAI;QACF,IAAI,MAAM,IAAA,yBAAkB,EAAC,MAAM,EAAE,mCAA0B,CAAC,EAAE;YAChE,0BAA0B;YAC1B,MAAM,MAAM,CAAC,mBAAmB,CAC9B,MAAM,EACN,UAAU,EACV,WAAW,EACX,iBAAiB,EACjB,MAAM,CACP,CAAC;SACH;aAAM;YACL,KAAK,MAAM,QAAQ,IAAI,MAAM,CAAC,SAAS,EAAE;gBACvC,yBAAyB;gBACzB,MAAM,MAAM,CAAC,YAAY,CACvB,IAAI,CAAC,qBAAqB,CAAC,MAAM,EAAE,QAAQ,CAAC,EAC5C,QAAQ,EACR,UAAU,CACX,CAAC;aACH;SACF;KACF;IAAC,OAAO,CAAC,EAAE;QACV,MAAM,YAAY,CAAC,CAAC,CAAC,CAAC;KACvB;IACD,OAAO,MAAM,IAAA,uCAAuB,EAAC,MAAM,EAAE,MAAM,CAAC,CAAC;AACvD,CAAC;AAlCD,0BAkCC;AAED;;;;;;;;GAQG;AACH,SAAS,YAAY,CAAC,CAAM;IAC1B,IAAI,CAAC,CAAC,CAAC,YAAY,KAAK,CAAC,EAAE;QACzB,OAAO,CAAC,CAAC;KACV;IAED;IACE,2BAA2B;IAC3B,CAAC,CAAC,OAAO,EAAE,QAAQ,CAAC,8BAA8B,CAAC;QACnD,CAAC,CAAC,OAAO,EAAE,QAAQ,CAAC,uCAAuC,CAAC,EAC5D;QACA,OAAO,IAAI,IAAI,CAAC,SAAS,CACvB,sDAAsD,CAAC,CAAC,OAAO,EAAE,CAClE,CAAC;KACH;IAED;IACE,+EAA+E;IAC/E,CAAC,CAAC,OAAO,EAAE,QAAQ,CAAC,wCAAwC,CAAC;QAC7D,gEAAgE;QAChE,CAAC,CAAC,OAAO,EAAE,QAAQ,CAAC,qBAAqB,CAAC,EAC1C;QACA,OAAO,IAAI,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC;KACtC;IAED,OAAO,CAAC,CAAC;AACX,CAAC;AAED,sEAAsE;AACtE,4EAA4E;AAC5E,4EAA4E;AAC5E,6EAA6E;AAC7E,+CAA+C;AACxC,KAAK,UAAU,mBAAmB,CACvC,WAA+B,EAC/B,YAAgC,EAChC,MAA0B,EAC1B,MAAc,EACd,YAA0B;IAE1B,IAAI,MAAc,CAAC;IACnB,IAAI,WAAW,KAAK,SAAS,EAAE;QAC7B,MAAM,GAAG;;;;;;;;;;;;uCAY0B,WAAW;;8BAEpB,WAAW;;;;;;;;gDAQO,CAAC;KAC9C;SAAM;QACL,oEAAoE;QACpE,mFAAmF;QACnF,+EAA+E;QAC/E,kFAAkF;QAClF,6EAA6E;QAC7E,oFAAoF;QACpF,6CAA6C;QAC7C,YAAY,GAAG,YAAY,IAAI,CAAC,CAAC;QACjC,MAAM,GAAG;;;;;;;;4BAQe,YAAY;;;;;;;;;;;;;;;;;;;;;gDAqBQ,CAAC;KAC9C;IAED,MAAM,gBAAgB,GAAG,IAAI,CAAC,IAAI,CAAC,MAAM,CAAC,OAAO,EAAE,mBAAmB,CAAC,CAAC;IACxE,EAAE,CAAC,aAAa,CAAC,gBAAgB,EAAE,MAAM,CAAC,CAAC;IAE3C,MAAM,IAAI,UAAU,CAAC,UAAU,CAC7B,MAAM,SAAS,CAAC,SAAS,CAAC,YAAY,CAAC,EACvC;QACE,kBAAkB;QAClB,QAAQ;QACR,OAAO;QACP,gBAAgB;QAChB,IAAI,CAAC,OAAO,CACV,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC,OAAO,EAAE,CAAC,EAC9B,OAAO,EACP,OAAO,EACP,YAAY,CACb;KACF,EACD,EAAE,GAAG,EAAE,EAAE,0BAA0B,EAAE,YAAY,CAAC,IAAI,EAAE,EAAE,CAC3D,CAAC,IAAI,EAAE,CAAC;AACX,CAAC;AA5FD,kDA4FC;AAEM,KAAK,UAAU,iBAAiB,CAAC,MAAc,EAAE,MAAc;IACpE,MAAM,CAAC,UAAU,CAAC,2BAA2B,CAAC,CAAC;IAE/C,MAAM,aAAa,GAAG,IAAI,CAAC,OAAO,CAAC,SAAS,EAAE,iBAAiB,CAAC,CAAC;IAEjE,IAAI;QACF,IAAI,OAAO,CAAC,QAAQ,KAAK,OAAO,EAAE;YAChC,MAAM,IAAI,UAAU,CAAC,UAAU,CAAC,MAAM,SAAS,CAAC,SAAS,CAAC,YAAY,CAAC,EAAE;gBACvE,IAAI,CAAC,IAAI,CAAC,aAAa,EAAE,mBAAmB,CAAC;aAC9C,CAAC,CAAC,IAAI,EAAE,CAAC;SACX;aAAM;YACL,MAAM,IAAI,UAAU,CAAC,UAAU,CAC7B,IAAI,CAAC,IAAI,CAAC,aAAa,EAAE,kBAAkB,CAAC,CAC7C,CAAC,IAAI,EAAE,CAAC;SACV;QACD,MAAM,MAAM,GAAG,0BAA0B,CAAC;QAC1C,IAAI,OAAO,CAAC,QAAQ,KAAK,OAAO,EAAE;YAChC,MAAM,IAAI,UAAU,CAAC,UAAU,CAAC,MAAM,SAAS,CAAC,SAAS,CAAC,IAAI,CAAC,EAAE;gBAC/D,IAAI;gBACJ,IAAI;gBACJ,IAAI,CAAC,IAAI,CAAC,aAAa,EAAE,MAAM,CAAC;gBAChC,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC,OAAO,EAAE,CAAC;aAC/B,CAAC,CAAC,IAAI,EAAE,CAAC;SACX;aAAM;YACL,MAAM,IAAI,UAAU,CAAC,UAAU,CAAC,MAAM,SAAS,CAAC,SAAS,CAAC,SAAS,CAAC,EAAE;gBACpE,IAAI;gBACJ,IAAI,CAAC,IAAI,CAAC,aAAa,EAAE,MAAM,CAAC;gBAChC,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC,OAAO,EAAE,CAAC;aAC/B,CAAC,CAAC,IAAI,EAAE,CAAC;SACX;KACF;IAAC,OAAO,CAAC,EAAE;QACV,MAAM,CAAC,QAAQ,EAAE,CAAC;QAClB,MAAM,CAAC,OAAO,CACZ,gFAAgF,CAAC,IAAI;YACnF,qGAAqG;YACrG,oGAAoG;YACpG,iDAAiD,CACpD,CAAC;QACF,OAAO;KACR;IACD,MAAM,CAAC,QAAQ,EAAE,CAAC;AACpB,CAAC;AAzCD,8CAyCC"}
{"version":3,"file":"init.js","sourceRoot":"","sources":["../src/init.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;;;;;;;;;AAAA,uCAAyB;AACzB,2CAA6B;AAE7B,yEAA2D;AAC3D,kEAAoD;AAEpD,gEAAkD;AAElD,qCAA2E;AAC3E,4DAA8C;AAI9C,mDAAwE;AACxE,6CAA+B;AAC/B,iCAA4C;AAE5C,IAAY,WAKX;AALD,WAAY,WAAW;IACrB,kCAAmB,CAAA;IACnB,8BAAe,CAAA;IACf,sCAAuB,CAAA;IACvB,oCAAqB,CAAA;AACvB,CAAC,EALW,WAAW,GAAX,mBAAW,KAAX,mBAAW,QAKtB;AAEM,KAAK,UAAU,UAAU,CAC9B,UAA8B,EAC9B,UAA4B,EAC5B,OAAe,EACf,OAA2B,EAC3B,iBAA2C,EAC3C,MAAc;IAOd,MAAM,CAAC,UAAU,CAAC,oBAAoB,CAAC,CAAC;IACxC,MAAM,EAAE,MAAM,EAAE,uBAAuB,EAAE,WAAW,EAAE,YAAY,EAAE,GAClE,MAAM,IAAA,oBAAW,EACf,UAAU,EACV,UAAU,EACV,OAAO,EACP,OAAO,EACP,iBAAiB,EACjB,MAAM,EACN,IAAI,CACL,CAAC;IACJ,MAAM,MAAM,CAAC,YAAY,EAAE,CAAC;IAC5B,MAAM,CAAC,QAAQ,EAAE,CAAC;IAClB,OAAO,EAAE,MAAM,EAAE,uBAAuB,EAAE,WAAW,EAAE,YAAY,EAAE,CAAC;AACxE,CAAC;AA3BD,gCA2BC;AAEM,KAAK,UAAU,UAAU,CAC9B,cAAkC,EAClC,YAAgC,EAChC,UAA8B,EAC9B,eAAmC,EACnC,UAA8B,EAC9B,UAA8B,EAC9B,kBAA2B,EAC3B,SAAkB,EAClB,iBAAyB,EACzB,iBAAyB,EACzB,UAAyB,EACzB,OAAe,EACf,MAAc,EACd,aAAqB,EACrB,aAAiC,EACjC,UAAoC,EACpC,iBAAoC,EACpC,MAAc;IAEd,MAAM,CAAC,UAAU,CAAC,6BAA6B,CAAC,CAAC;IACjD,MAAM,MAAM,GAAG,MAAM,WAAW,CAAC,UAAU,CACzC,cAAc,EACd,YAAY,EACZ,UAAU,EACV,eAAe,EACf,UAAU,EACV,UAAU,EACV,kBAAkB,EAClB,SAAS,EACT,iBAAiB,EACjB,iBAAiB,EACjB,UAAU,EACV,OAAO,EACP,MAAM,EACN,aAAa,EACb,aAAa,EACb,UAAU,EACV,iBAAiB,EACjB,MAAM,CACP,CAAC;IACF,aAAa,CAAC,uBAAuB,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IACtD,MAAM,CAAC,QAAQ,EAAE,CAAC;IAClB,OAAO,MAAM,CAAC;AAChB,CAAC;AA5CD,gCA4CC;AAEM,KAAK,UAAU,OAAO,CAC3B,MAAc,EACd,MAA0B,EAC1B,UAAkB,EAClB,WAA+B,EAC/B,iBAAoC,EACpC,MAAc;IAEd,EAAE,CAAC,SAAS,CAAC,MAAM,CAAC,UAAU,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;IAErD,IAAI;QACF,IAAI,MAAM,IAAA,yBAAkB,EAAC,MAAM,EAAE,mCAA0B,CAAC,EAAE;YAChE,0BAA0B;YAC1B,MAAM,MAAM,CAAC,mBAAmB,CAC9B,MAAM,EACN,UAAU,EACV,WAAW,EACX,iBAAiB,EACjB,MAAM,CACP,CAAC;SACH;aAAM;YACL,KAAK,MAAM,QAAQ,IAAI,MAAM,CAAC,SAAS,EAAE;gBACvC,yBAAyB;gBACzB,MAAM,MAAM,CAAC,YAAY,CACvB,IAAI,CAAC,qBAAqB,CAAC,MAAM,EAAE,QAAQ,CAAC,EAC5C,QAAQ,EACR,UAAU,CACX,CAAC;aACH;SACF;KACF;IAAC,OAAO,CAAC,EAAE;QACV,MAAM,YAAY,CAAC,CAAC,CAAC,CAAC;KACvB;IACD,OAAO,MAAM,IAAA,uCAAuB,EAAC,MAAM,EAAE,MAAM,CAAC,CAAC;AACvD,CAAC;AAlCD,0BAkCC;AAED;;;;;;;;GAQG;AACH,SAAS,YAAY,CAAC,CAAM;IAC1B,IAAI,CAAC,CAAC,CAAC,YAAY,KAAK,CAAC,EAAE;QACzB,OAAO,CAAC,CAAC;KACV;IAED;IACE,2BAA2B;IAC3B,CAAC,CAAC,OAAO,EAAE,QAAQ,CAAC,8BAA8B,CAAC;QACnD,CAAC,CAAC,OAAO,EAAE,QAAQ,CAAC,uCAAuC,CAAC,EAC5D;QACA,OAAO,IAAI,IAAI,CAAC,SAAS,CACvB,sDAAsD,CAAC,CAAC,OAAO,EAAE,CAClE,CAAC;KACH;IAED;IACE,+EAA+E;IAC/E,CAAC,CAAC,OAAO,EAAE,QAAQ,CAAC,wCAAwC,CAAC;QAC7D,gEAAgE;QAChE,CAAC,CAAC,OAAO,EAAE,QAAQ,CAAC,qBAAqB,CAAC,EAC1C;QACA,OAAO,IAAI,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC;KACtC;IAED,OAAO,CAAC,CAAC;AACX,CAAC;AAED,sEAAsE;AACtE,4EAA4E;AAC5E,4EAA4E;AAC5E,6EAA6E;AAC7E,+CAA+C;AACxC,KAAK,UAAU,mBAAmB,CACvC,WAA+B,EAC/B,YAAgC,EAChC,MAA0B,EAC1B,MAAc,EACd,YAA0B;IAE1B,IAAI,MAAc,CAAC;IACnB,IAAI,WAAW,KAAK,SAAS,EAAE;QAC7B,MAAM,GAAG;;;;;;;;;;;;uCAY0B,WAAW;;8BAEpB,WAAW;;;;;;;;gDAQO,CAAC;KAC9C;SAAM;QACL,oEAAoE;QACpE,mFAAmF;QACnF,+EAA+E;QAC/E,kFAAkF;QAClF,6EAA6E;QAC7E,oFAAoF;QACpF,6CAA6C;QAC7C,YAAY,GAAG,YAAY,IAAI,CAAC,CAAC;QACjC,MAAM,GAAG;;;;;;;;4BAQe,YAAY;;;;;;;;;;;;;;;;;;;;;gDAqBQ,CAAC;KAC9C;IAED,MAAM,gBAAgB,GAAG,IAAI,CAAC,IAAI,CAAC,MAAM,CAAC,OAAO,EAAE,mBAAmB,CAAC,CAAC;IACxE,EAAE,CAAC,aAAa,CAAC,gBAAgB,EAAE,MAAM,CAAC,CAAC;IAE3C,MAAM,IAAI,UAAU,CAAC,UAAU,CAC7B,MAAM,SAAS,CAAC,SAAS,CAAC,YAAY,CAAC,EACvC;QACE,kBAAkB;QAClB,QAAQ;QACR,OAAO;QACP,gBAAgB;QAChB,IAAI,CAAC,OAAO,CACV,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC,OAAO,EAAE,CAAC,EAC9B,OAAO,EACP,OAAO,EACP,YAAY,CACb;KACF,EACD,EAAE,GAAG,EAAE,EAAE,0BAA0B,EAAE,YAAY,CAAC,IAAI,EAAE,EAAE,CAC3D,CAAC,IAAI,EAAE,CAAC;AACX,CAAC;AA5FD,kDA4FC;AAEM,KAAK,UAAU,iBAAiB,CAAC,MAAc,EAAE,MAAc;IACpE,MAAM,CAAC,UAAU,CAAC,2BAA2B,CAAC,CAAC;IAE/C,MAAM,aAAa,GAAG,IAAI,CAAC,OAAO,CAAC,SAAS,EAAE,iBAAiB,CAAC,CAAC;IAEjE,IAAI;QACF,IAAI,OAAO,CAAC,QAAQ,KAAK,OAAO,EAAE;YAChC,MAAM,IAAI,UAAU,CAAC,UAAU,CAAC,MAAM,SAAS,CAAC,SAAS,CAAC,YAAY,CAAC,EAAE;gBACvE,IAAI,CAAC,IAAI,CAAC,aAAa,EAAE,mBAAmB,CAAC;aAC9C,CAAC,CAAC,IAAI,EAAE,CAAC;SACX;aAAM;YACL,MAAM,IAAI,UAAU,CAAC,UAAU,CAC7B,IAAI,CAAC,IAAI,CAAC,aAAa,EAAE,kBAAkB,CAAC,CAC7C,CAAC,IAAI,EAAE,CAAC;SACV;QACD,MAAM,MAAM,GAAG,0BAA0B,CAAC;QAC1C,IAAI,OAAO,CAAC,QAAQ,KAAK,OAAO,EAAE;YAChC,MAAM,IAAI,UAAU,CAAC,UAAU,CAAC,MAAM,SAAS,CAAC,SAAS,CAAC,IAAI,CAAC,EAAE;gBAC/D,IAAI;gBACJ,IAAI;gBACJ,IAAI,CAAC,IAAI,CAAC,aAAa,EAAE,MAAM,CAAC;gBAChC,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC,OAAO,EAAE,CAAC;aAC/B,CAAC,CAAC,IAAI,EAAE,CAAC;SACX;aAAM;YACL,MAAM,IAAI,UAAU,CAAC,UAAU,CAAC,MAAM,SAAS,CAAC,SAAS,CAAC,SAAS,CAAC,EAAE;gBACpE,IAAI;gBACJ,IAAI,CAAC,IAAI,CAAC,aAAa,EAAE,MAAM,CAAC;gBAChC,IAAI,CAAC,OAAO,CAAC,MAAM,CAAC,OAAO,EAAE,CAAC;aAC/B,CAAC,CAAC,IAAI,EAAE,CAAC;SACX;KACF;IAAC,OAAO,CAAC,EAAE;QACV,MAAM,CAAC,QAAQ,EAAE,CAAC;QAClB,MAAM,CAAC,OAAO,CACZ,gFAAgF,CAAC,IAAI;YACnF,qGAAqG;YACrG,oGAAoG;YACpG,iDAAiD,CACpD,CAAC;QACF,OAAO;KACR;IACD,MAAM,CAAC,QAAQ,EAAE,CAAC;AACpB,CAAC;AAzCD,8CAyCC"}
+1 -2
View File
@@ -1,6 +1,6 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.isScannedLanguage = exports.isTracedLanguage = exports.parseLanguage = exports.resolveAlias = exports.KOTLIN_SWIFT_BYPASS = exports.LANGUAGE_ALIASES = exports.Language = void 0;
exports.isScannedLanguage = exports.isTracedLanguage = exports.parseLanguage = exports.resolveAlias = exports.LANGUAGE_ALIASES = exports.Language = void 0;
// All the languages supported by CodeQL
var Language;
(function (Language) {
@@ -21,7 +21,6 @@ exports.LANGUAGE_ALIASES = {
kotlin: Language.java,
typescript: Language.javascript,
};
exports.KOTLIN_SWIFT_BYPASS = ["kotlin", "swift"];
function resolveAlias(lang) {
return exports.LANGUAGE_ALIASES[lang] || lang;
}
+1 -1
View File
@@ -1 +1 @@
{"version":3,"file":"languages.js","sourceRoot":"","sources":["../src/languages.ts"],"names":[],"mappings":";;;AAAA,wCAAwC;AACxC,IAAY,QASX;AATD,WAAY,QAAQ;IAClB,6BAAiB,CAAA;IACjB,uBAAW,CAAA;IACX,qBAAS,CAAA;IACT,yBAAa,CAAA;IACb,qCAAyB,CAAA;IACzB,6BAAiB,CAAA;IACjB,yBAAa,CAAA;IACb,2BAAe,CAAA;AACjB,CAAC,EATW,QAAQ,GAAR,gBAAQ,KAAR,gBAAQ,QASnB;AAED,iCAAiC;AACpB,QAAA,gBAAgB,GAAiC;IAC5D,CAAC,EAAE,QAAQ,CAAC,GAAG;IACf,KAAK,EAAE,QAAQ,CAAC,GAAG;IACnB,IAAI,EAAE,QAAQ,CAAC,MAAM;IACrB,MAAM,EAAE,QAAQ,CAAC,IAAI;IACrB,UAAU,EAAE,QAAQ,CAAC,UAAU;CAChC,CAAC;AAIW,QAAA,mBAAmB,GAAG,CAAC,QAAQ,EAAE,OAAO,CAAC,CAAC;AAEvD,SAAgB,YAAY,CAAC,IAAqB;IAChD,OAAO,wBAAgB,CAAC,IAAI,CAAC,IAAI,IAAI,CAAC;AACxC,CAAC;AAFD,oCAEC;AAED;;;;;;;;;GASG;AACH,SAAgB,aAAa,CAAC,QAAgB;IAC5C,0BAA0B;IAC1B,QAAQ,GAAG,QAAQ,CAAC,IAAI,EAAE,CAAC,WAAW,EAAE,CAAC;IAEzC,6BAA6B;IAC7B,IAAI,QAAQ,IAAI,QAAQ,EAAE;QACxB,OAAO,QAAoB,CAAC;KAC7B;IAED,iEAAiE;IACjE,oCAAoC;IACpC,IAAI,QAAQ,IAAI,wBAAgB,EAAE;QAChC,OAAO,QAAQ,CAAC;KACjB;IAED,OAAO,SAAS,CAAC;AACnB,CAAC;AAhBD,sCAgBC;AAED,SAAgB,gBAAgB,CAAC,QAAkB;IACjD,OAAO;QACL,QAAQ,CAAC,GAAG;QACZ,QAAQ,CAAC,MAAM;QACf,QAAQ,CAAC,EAAE;QACX,QAAQ,CAAC,IAAI;QACb,QAAQ,CAAC,KAAK;KACf,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAC;AACvB,CAAC;AARD,4CAQC;AAED,SAAgB,iBAAiB,CAAC,QAAkB;IAClD,OAAO,CAAC,gBAAgB,CAAC,QAAQ,CAAC,CAAC;AACrC,CAAC;AAFD,8CAEC"}
{"version":3,"file":"languages.js","sourceRoot":"","sources":["../src/languages.ts"],"names":[],"mappings":";;;AAAA,wCAAwC;AACxC,IAAY,QASX;AATD,WAAY,QAAQ;IAClB,6BAAiB,CAAA;IACjB,uBAAW,CAAA;IACX,qBAAS,CAAA;IACT,yBAAa,CAAA;IACb,qCAAyB,CAAA;IACzB,6BAAiB,CAAA;IACjB,yBAAa,CAAA;IACb,2BAAe,CAAA;AACjB,CAAC,EATW,QAAQ,GAAR,gBAAQ,KAAR,gBAAQ,QASnB;AAED,iCAAiC;AACpB,QAAA,gBAAgB,GAAiC;IAC5D,CAAC,EAAE,QAAQ,CAAC,GAAG;IACf,KAAK,EAAE,QAAQ,CAAC,GAAG;IACnB,IAAI,EAAE,QAAQ,CAAC,MAAM;IACrB,MAAM,EAAE,QAAQ,CAAC,IAAI;IACrB,UAAU,EAAE,QAAQ,CAAC,UAAU;CAChC,CAAC;AAIF,SAAgB,YAAY,CAAC,IAAqB;IAChD,OAAO,wBAAgB,CAAC,IAAI,CAAC,IAAI,IAAI,CAAC;AACxC,CAAC;AAFD,oCAEC;AAED;;;;;;;;;GASG;AACH,SAAgB,aAAa,CAAC,QAAgB;IAC5C,0BAA0B;IAC1B,QAAQ,GAAG,QAAQ,CAAC,IAAI,EAAE,CAAC,WAAW,EAAE,CAAC;IAEzC,6BAA6B;IAC7B,IAAI,QAAQ,IAAI,QAAQ,EAAE;QACxB,OAAO,QAAoB,CAAC;KAC7B;IAED,iEAAiE;IACjE,oCAAoC;IACpC,IAAI,QAAQ,IAAI,wBAAgB,EAAE;QAChC,OAAO,QAAQ,CAAC;KACjB;IAED,OAAO,SAAS,CAAC;AACnB,CAAC;AAhBD,sCAgBC;AAED,SAAgB,gBAAgB,CAAC,QAAkB;IACjD,OAAO;QACL,QAAQ,CAAC,GAAG;QACZ,QAAQ,CAAC,MAAM;QACf,QAAQ,CAAC,EAAE;QACX,QAAQ,CAAC,IAAI;QACb,QAAQ,CAAC,KAAK;KACf,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAC;AACvB,CAAC;AARD,4CAQC;AAED,SAAgB,iBAAiB,CAAC,QAAkB;IAClD,OAAO,CAAC,gBAAgB,CAAC,QAAQ,CAAC,CAAC;AACrC,CAAC;AAFD,8CAEC"}
+167 -115
View File
@@ -26,7 +26,7 @@ var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.setupCodeQLBundle = exports.getCodeQLURLVersion = exports.downloadCodeQL = exports.getCodeQLSource = exports.convertToSemVer = exports.getBundleVersionFromUrl = exports.tryFindCliVersionDotcomOnly = exports.findCodeQLBundleTagDotcomOnly = exports.getCodeQLActionRepository = exports.CODEQL_DEFAULT_ACTION_REPOSITORY = void 0;
exports.setupCodeQLBundle = exports.getCodeQLURLVersion = exports.downloadCodeQL = exports.tryGetFallbackToolcacheVersion = exports.getCodeQLSource = exports.convertToSemVer = exports.tryGetBundleVersionFromUrl = exports.tryFindCliVersionDotcomOnly = exports.findCodeQLBundleTagDotcomOnly = exports.getCodeQLActionRepository = exports.CODEQL_DEFAULT_ACTION_REPOSITORY = void 0;
const fs = __importStar(require("fs"));
const path = __importStar(require("path"));
const perf_hooks_1 = require("perf_hooks");
@@ -211,21 +211,30 @@ async function getCodeQLBundleDownloadURL(tagName, apiDetails, variant, logger)
}
return `https://github.com/${exports.CODEQL_DEFAULT_ACTION_REPOSITORY}/releases/download/${tagName}/${codeQLBundleName}`;
}
function getBundleVersionFromTagName(tagName) {
function tryGetBundleVersionFromTagName(tagName, logger) {
const match = tagName.match(/^codeql-bundle-(.*)$/);
if (match === null || match.length < 2) {
throw new Error(`Malformed bundle tag name: ${tagName}. Bundle version could not be inferred`);
logger.debug(`Could not determine bundle version from tag ${tagName}.`);
return undefined;
}
return match[1];
}
function getBundleVersionFromUrl(url) {
function tryGetTagNameFromUrl(url, logger) {
const match = url.match(/\/(codeql-bundle-.*)\//);
if (match === null || match.length < 2) {
throw new Error(`Malformed tools url: ${url}. Bundle version could not be inferred`);
logger.debug(`Could not determine tag name for URL ${url}.`);
return undefined;
}
return getBundleVersionFromTagName(match[1]);
return match[1];
}
exports.getBundleVersionFromUrl = getBundleVersionFromUrl;
function tryGetBundleVersionFromUrl(url, logger) {
const tagName = tryGetTagNameFromUrl(url, logger);
if (tagName === undefined) {
return undefined;
}
return tryGetBundleVersionFromTagName(tagName, logger);
}
exports.tryGetBundleVersionFromUrl = tryGetBundleVersionFromUrl;
function convertToSemVer(version, logger) {
if (!semver.valid(version)) {
logger.debug(`Bundle version ${version} is not in SemVer format. Will treat it as pre-release 0.0.0-${version}.`);
@@ -238,18 +247,10 @@ function convertToSemVer(version, logger) {
return s;
}
exports.convertToSemVer = convertToSemVer;
async function getOrFindBundleTagName(version, logger) {
if (version.variant === util.GitHubVariant.DOTCOM) {
return await findCodeQLBundleTagDotcomOnly(version.cliVersion, logger);
}
else {
return version.tagName;
}
}
/**
* Look for a version of the CodeQL tools in the cache which could override the requested CLI version.
*/
async function findOverridingToolsInCache(requestedCliVersion, logger) {
async function findOverridingToolsInCache(humanReadableVersion, logger) {
const candidates = toolcache
.findAllVersions("CodeQL")
.filter(util_1.isGoodVersion)
@@ -260,7 +261,7 @@ async function findOverridingToolsInCache(requestedCliVersion, logger) {
.filter(({ folder }) => fs.existsSync(path.join(folder, "pinned-version")));
if (candidates.length === 1) {
const candidate = candidates[0];
logger.debug(`CodeQL tools version ${candidate.version} in toolcache overriding version ${requestedCliVersion}.`);
logger.debug(`CodeQL tools version ${candidate.version} in toolcache overriding version ${humanReadableVersion}.`);
return {
codeqlFolder: candidate.folder,
sourceType: "toolcache",
@@ -276,7 +277,7 @@ async function findOverridingToolsInCache(requestedCliVersion, logger) {
}
return undefined;
}
async function getCodeQLSource(toolsInput, bypassToolcache, defaultCliVersion, apiDetails, variant, logger) {
async function getCodeQLSource(toolsInput, defaultCliVersion, apiDetails, variant, logger) {
if (toolsInput && toolsInput !== "latest" && !toolsInput.startsWith("http")) {
return {
codeqlTarPath: toolsInput,
@@ -284,124 +285,166 @@ async function getCodeQLSource(toolsInput, bypassToolcache, defaultCliVersion, a
toolsVersion: "local",
};
}
const forceLatestReason =
// We use the special value of 'latest' to prioritize the version in the
// defaults over any pinned cached version.
toolsInput === "latest"
? '"tools: latest" was requested'
: // If the user hasn't requested a particular CodeQL version, then bypass
// the toolcache when the appropriate feature is enabled. This
// allows us to quickly rollback a broken bundle that has made its way
// into the toolcache.
toolsInput === undefined && bypassToolcache
? "a specific version of the CodeQL tools was not requested and the bypass toolcache feature is enabled"
: undefined;
const forceLatest = forceLatestReason !== undefined;
if (forceLatest) {
logger.debug(`Forcing the latest version of the CodeQL tools since ${forceLatestReason}.`);
}
/**
* The requested version is:
* Whether the tools shipped with the Action, i.e. those in `defaults.json`, have been forced.
*
* 1. The one in `defaults.json`, if forceLatest is true.
* 2. The version specified by the tools input URL, if one was provided.
* 3. The default CLI version, otherwise.
* We include a `variant` property to let us verify using the type system that
* `tagName` is only undefined when the variant is Dotcom. This lets us ensure
* that we can always compute `tagName`, either by using the existing tag name
* on enterprise instances, or calling `findCodeQLBundleTagDotcomOnly` on
* Dotcom.
* We use the special value of 'latest' to prioritize the version in `defaults.json` over the
* version specified by the feature flags on Dotcom and over any pinned cached version on
* Enterprise Server.
*/
const requestedVersion = forceLatest
? // case 1
{
cliVersion: defaults.cliVersion,
syntheticCliVersion: defaults.cliVersion,
tagName: defaults.bundleVersion,
variant,
}
: toolsInput !== undefined
? // case 2
{
syntheticCliVersion: convertToSemVer(getBundleVersionFromUrl(toolsInput), logger),
tagName: `codeql-bundle-${getBundleVersionFromUrl(toolsInput)}`,
url: toolsInput,
variant,
}
: // case 3
{
...defaultCliVersion,
syntheticCliVersion: defaultCliVersion.cliVersion,
};
// If we find the specified version, we always use that.
let codeqlFolder = toolcache.find("CodeQL", requestedVersion.syntheticCliVersion);
let tagName = requestedVersion["tagName"];
if (!codeqlFolder) {
logger.debug("Didn't find a version of the CodeQL tools in the toolcache with a version number " +
`exactly matching ${requestedVersion.syntheticCliVersion}.`);
if (requestedVersion.cliVersion) {
const forceShippedTools = toolsInput === "latest";
if (forceShippedTools) {
logger.info("Overriding the version of the CodeQL tools by the version shipped with the Action since " +
`"tools: latest" was requested.`);
}
/** CLI version number, for example 2.12.1. */
let cliVersion;
/** Tag name of the CodeQL bundle, for example `codeql-bundle-20230120`. */
let tagName;
/**
* URL of the CodeQL bundle.
*
* This does not always include a tag name.
*/
let url;
if (forceShippedTools) {
cliVersion = defaults.cliVersion;
tagName = defaults.bundleVersion;
}
else if (toolsInput !== undefined) {
// If a tools URL was provided, then use that.
tagName = tryGetTagNameFromUrl(toolsInput, logger);
url = toolsInput;
}
else {
// Otherwise, use the default CLI version passed in.
cliVersion = defaultCliVersion.cliVersion;
tagName = defaultCliVersion["tagName"];
}
const bundleVersion = tagName && tryGetBundleVersionFromTagName(tagName, logger);
const humanReadableVersion = cliVersion ??
(bundleVersion && convertToSemVer(bundleVersion, logger)) ??
tagName ??
url ??
"unknown";
logger.debug("Attempting to obtain CodeQL tools. " +
`CLI version: ${cliVersion ?? "unknown"}, ` +
`bundle tag name: ${tagName ?? "unknown"}, ` +
`URL: ${url ?? "unspecified"}.`);
let codeqlFolder;
if (cliVersion) {
// If we find the specified CLI version, we always use that.
codeqlFolder = toolcache.find("CodeQL", cliVersion);
// Fall back to matching `x.y.z-<tagName>`.
if (!codeqlFolder) {
logger.debug("Didn't find a version of the CodeQL tools in the toolcache with a version number " +
`exactly matching ${cliVersion}.`);
const allVersions = toolcache.findAllVersions("CodeQL");
logger.debug(`Found the following versions of the CodeQL tools in the toolcache: ${JSON.stringify(allVersions)}.`);
// If there is exactly one version of the CodeQL tools in the toolcache, and that version is
// the form `x.y.z-<tagName>`, then use it.
const candidateVersions = allVersions.filter((version) => version.startsWith(`${requestedVersion.cliVersion}-`));
const candidateVersions = allVersions.filter((version) => version.startsWith(`${cliVersion}-`));
if (candidateVersions.length === 1) {
logger.debug("Exactly one candidate version found, using that.");
logger.debug(`Exactly one version of the CodeQL tools starting with ${cliVersion} found in the ` +
"toolcache, using that.");
codeqlFolder = toolcache.find("CodeQL", candidateVersions[0]);
}
else if (candidateVersions.length === 0) {
logger.debug(`Didn't find any versions of the CodeQL tools starting with ${cliVersion} ` +
`in the toolcache. Trying next fallback method.`);
}
else {
logger.debug("Did not find exactly one version of the CodeQL tools starting with the requested version.");
logger.warning(`Found ${candidateVersions.length} versions of the CodeQL tools starting with ` +
`${cliVersion} in the toolcache, but at most one was expected.`);
logger.debug("Trying next fallback method.");
}
}
}
if (!codeqlFolder && requestedVersion.cliVersion) {
// Fall back to accepting a `0.0.0-<bundleVersion>` version if we didn't find the
// `x.y.z` version. This is to support old versions of the toolcache.
//
// If we are on Dotcom, we will make an HTTP request to the Releases API here
// to find the tag name for the requested version.
tagName =
tagName || (await getOrFindBundleTagName(requestedVersion, logger));
const fallbackVersion = convertToSemVer(getBundleVersionFromTagName(tagName), logger);
logger.debug(`Computed a fallback toolcache version number of ${fallbackVersion} for CodeQL tools version ` +
`${requestedVersion.cliVersion}.`);
codeqlFolder = toolcache.find("CodeQL", fallbackVersion);
// Fall back to matching `0.0.0-<bundleVersion>`.
if (!codeqlFolder && (cliVersion || tagName)) {
if (cliVersion || tagName) {
const fallbackVersion = await tryGetFallbackToolcacheVersion(cliVersion, tagName, variant, logger);
if (fallbackVersion) {
codeqlFolder = toolcache.find("CodeQL", fallbackVersion);
}
else {
logger.debug("Could not determine a fallback toolcache version number for CodeQL tools version " +
`${humanReadableVersion}.`);
}
}
else {
logger.debug("Both the CLI version and the bundle version are unknown, so we will not be able to find " +
"the requested version of the CodeQL tools in the toolcache.");
}
}
if (codeqlFolder) {
logger.info(`Found CodeQL tools version ${humanReadableVersion} in the toolcache.`);
}
else {
logger.info(`Did not find CodeQL tools version ${humanReadableVersion} in the toolcache.`);
}
if (codeqlFolder) {
return {
codeqlFolder,
sourceType: "toolcache",
toolsVersion: requestedVersion.syntheticCliVersion,
toolsVersion: cliVersion ?? humanReadableVersion,
};
}
logger.debug(`Did not find CodeQL tools version ${requestedVersion.syntheticCliVersion} in the toolcache.`);
// If we don't find the requested version on Enterprise, we may allow a
// different version to save download time if the version hasn't been
// specified explicitly (in which case we always honor it).
if (variant !== util.GitHubVariant.DOTCOM && !forceLatest && !toolsInput) {
const result = await findOverridingToolsInCache(requestedVersion.syntheticCliVersion, logger);
if (variant !== util.GitHubVariant.DOTCOM &&
!forceShippedTools &&
!toolsInput) {
const result = await findOverridingToolsInCache(humanReadableVersion, logger);
if (result !== undefined) {
return result;
}
}
if (!url) {
if (!tagName && cliVersion && variant === util.GitHubVariant.DOTCOM) {
tagName = await findCodeQLBundleTagDotcomOnly(cliVersion, logger);
}
else if (!tagName) {
throw new Error(`Could not obtain the requested version (${humanReadableVersion}) of the CodeQL tools ` +
"since we could not compute the tag name.");
}
url = await getCodeQLBundleDownloadURL(tagName, apiDetails, variant, logger);
}
return {
cliVersion: requestedVersion.cliVersion || undefined,
codeqlURL: requestedVersion["url"] ||
(await getCodeQLBundleDownloadURL(tagName ||
// The check on `requestedVersion.tagName` is redundant but lets us
// use the property that if we don't know `requestedVersion.tagName`,
// then we must know `requestedVersion.cliVersion`. This property is
// required by the type of `getOrFindBundleTagName`.
(requestedVersion.tagName !== undefined
? requestedVersion.tagName
: await getOrFindBundleTagName(requestedVersion, logger)), apiDetails, variant, logger)),
bundleVersion: tagName && tryGetBundleVersionFromTagName(tagName, logger),
cliVersion,
codeqlURL: url,
sourceType: "download",
toolsVersion: requestedVersion.syntheticCliVersion,
toolsVersion: cliVersion ?? humanReadableVersion,
};
}
exports.getCodeQLSource = getCodeQLSource;
async function downloadCodeQL(codeqlURL, maybeCliVersion, apiDetails, variant, tempDir, logger) {
/**
* Gets a fallback version number to use when looking for CodeQL in the toolcache if we didn't find
* the `x.y.z` version. This is to support old versions of the toolcache.
*/
async function tryGetFallbackToolcacheVersion(cliVersion, tagName, variant, logger) {
//
// If we are on Dotcom, we will make an HTTP request to the Releases API here
// to find the tag name for the requested version.
if (cliVersion && !tagName && variant === util.GitHubVariant.DOTCOM) {
tagName = await findCodeQLBundleTagDotcomOnly(cliVersion, logger);
}
if (!tagName) {
return undefined;
}
const bundleVersion = tryGetBundleVersionFromTagName(tagName, logger);
if (!bundleVersion) {
return undefined;
}
const fallbackVersion = convertToSemVer(bundleVersion, logger);
logger.debug(`Computed a fallback toolcache version number of ${fallbackVersion} for CodeQL version ` +
`${cliVersion ?? tagName}.`);
return fallbackVersion;
}
exports.tryGetFallbackToolcacheVersion = tryGetFallbackToolcacheVersion;
async function downloadCodeQL(codeqlURL, maybeBundleVersion, maybeCliVersion, apiDetails, variant, tempDir, logger) {
const parsedCodeQLURL = new URL(codeqlURL);
const searchParams = new URLSearchParams(parsedCodeQLURL.search);
const headers = {
@@ -430,12 +473,22 @@ async function downloadCodeQL(codeqlURL, maybeCliVersion, apiDetails, variant, t
const toolsDownloadDurationMs = Math.round(perf_hooks_1.performance.now() - toolsDownloadStart);
logger.debug(`CodeQL bundle download to ${codeqlPath} complete.`);
const codeqlExtracted = await toolcache.extractTar(codeqlPath);
const bundleVersion = getBundleVersionFromUrl(codeqlURL);
const bundleVersion = maybeBundleVersion ?? tryGetBundleVersionFromUrl(codeqlURL, logger);
if (bundleVersion === undefined) {
logger.debug("Could not cache CodeQL tools because we could not determine the bundle version from the " +
`URL ${codeqlURL}.`);
return {
toolsVersion: maybeCliVersion ?? "unknown",
codeqlFolder: codeqlExtracted,
toolsDownloadDurationMs,
};
}
// Try to compute the CLI version for this bundle
const cliVersion = maybeCliVersion ||
(variant === util.GitHubVariant.DOTCOM &&
(await tryFindCliVersionDotcomOnly(`codeql-bundle-${bundleVersion}`, logger))) ||
undefined;
if (maybeCliVersion === undefined &&
variant === util.GitHubVariant.DOTCOM &&
codeqlURL.includes(`/${exports.CODEQL_DEFAULT_ACTION_REPOSITORY}/`)) {
maybeCliVersion = await tryFindCliVersionDotcomOnly(`codeql-bundle-${bundleVersion}`, logger);
}
// Include both the CLI version and the bundle version in the toolcache version number. That way
// if the user requests the same URL again, we can get it from the cache without having to call
// any of the Releases API.
@@ -445,11 +498,11 @@ async function downloadCodeQL(codeqlURL, maybeCliVersion, apiDetails, variant, t
// CLI release. In principle, it should be enough to just check that the CLI version isn't a
// pre-release, but the version numbers of CodeQL nightlies have the format `x.y.z+<timestamp>`,
// and we don't want these nightlies to override stable CLI versions in the toolcache.
const toolcacheVersion = cliVersion && cliVersion.match(/^[0-9]+\.[0-9]+\.[0-9]+$/)
? `${cliVersion}-${bundleVersion}`
const toolcacheVersion = maybeCliVersion?.match(/^[0-9]+\.[0-9]+\.[0-9]+$/)
? `${maybeCliVersion}-${bundleVersion}`
: convertToSemVer(bundleVersion, logger);
return {
toolsVersion: cliVersion || toolcacheVersion,
toolsVersion: maybeCliVersion ?? toolcacheVersion,
codeqlFolder: await toolcache.cacheDir(codeqlExtracted, "CodeQL", toolcacheVersion),
toolsDownloadDurationMs,
};
@@ -470,15 +523,14 @@ exports.getCodeQLURLVersion = getCodeQLURLVersion;
* @param apiDetails
* @param tempDir
* @param variant
* @param bypassToolcache
* @param defaultCliVersion
* @param logger
* @param checkVersion Whether to check that CodeQL CLI meets the minimum
* version requirement. Must be set to true outside tests.
* @returns the path to the extracted bundle, and the version of the tools
*/
async function setupCodeQLBundle(toolsInput, apiDetails, tempDir, variant, bypassToolcache, defaultCliVersion, logger) {
const source = await getCodeQLSource(toolsInput, bypassToolcache, defaultCliVersion, apiDetails, variant, logger);
async function setupCodeQLBundle(toolsInput, apiDetails, tempDir, variant, defaultCliVersion, logger) {
const source = await getCodeQLSource(toolsInput, defaultCliVersion, apiDetails, variant, logger);
let codeqlFolder;
let toolsVersion = source.toolsVersion;
let toolsDownloadDurationMs;
@@ -494,7 +546,7 @@ async function setupCodeQLBundle(toolsInput, apiDetails, tempDir, variant, bypas
toolsSource = init_1.ToolsSource.Toolcache;
break;
case "download": {
const result = await downloadCodeQL(source.codeqlURL, source.cliVersion, apiDetails, variant, tempDir, logger);
const result = await downloadCodeQL(source.codeqlURL, source.bundleVersion, source.cliVersion, apiDetails, variant, tempDir, logger);
toolsVersion = result.toolsVersion;
codeqlFolder = result.codeqlFolder;
toolsDownloadDurationMs = result.toolsDownloadDurationMs;
File diff suppressed because one or more lines are too long
+2 -1
View File
@@ -293,7 +293,8 @@ async function waitForProcessing(repositoryNwo, sarifID, logger, options = {
if (Date.now() >
statusCheckingStarted + STATUS_CHECK_TIMEOUT_MILLISECONDS) {
// If the analysis hasn't finished processing in the allotted time, we continue anyway rather than failing.
// It's possible the analysis will eventually finish processing, but it's not worth spending more Actions time waiting.
// It's possible the analysis will eventually finish processing, but it's not worth spending more
// Actions time waiting.
logger.warning("Timed out waiting for analysis to finish processing. Continuing.");
break;
}
File diff suppressed because one or more lines are too long
Generated
+1 -41
View File
@@ -26,7 +26,7 @@ var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.parseMatrixInput = exports.shouldBypassToolcache = exports.isHostedRunner = exports.checkForTimeout = exports.withTimeout = exports.tryGetFolderBytes = exports.listFolder = exports.doesDirectoryExist = exports.logCodeScanningConfigInCli = exports.useCodeScanningConfigInCli = exports.isInTestMode = exports.getMlPoweredJsQueriesStatus = exports.getMlPoweredJsQueriesPack = exports.ML_POWERED_JS_QUERIES_PACK_NAME = exports.isGoodVersion = exports.delay = exports.bundleDb = exports.codeQlVersionAbove = exports.getCachedCodeQlVersion = exports.cacheCodeQlVersion = exports.isHTTPError = exports.UserError = exports.HTTPError = exports.getRequiredEnvParam = exports.enrichEnvironment = exports.initializeEnvironment = exports.EnvVar = exports.assertNever = exports.apiVersionInRange = exports.DisallowedAPIVersionReason = exports.checkGitHubVersionInRange = exports.getGitHubVersion = exports.GitHubVariant = exports.parseGitHubUrl = exports.getCodeQLDatabasePath = exports.getThreadsFlag = exports.getThreadsFlagValue = exports.getAddSnippetsFlag = exports.getMemoryFlag = exports.getMemoryFlagValue = exports.withTmpDir = exports.getToolNames = exports.getExtraOptionsEnvParam = exports.DID_AUTOBUILD_GO_ENV_VAR_NAME = exports.DEFAULT_DEBUG_DATABASE_NAME = exports.DEFAULT_DEBUG_ARTIFACT_NAME = exports.GITHUB_DOTCOM_URL = void 0;
exports.parseMatrixInput = exports.isHostedRunner = exports.checkForTimeout = exports.withTimeout = exports.tryGetFolderBytes = exports.listFolder = exports.doesDirectoryExist = exports.logCodeScanningConfigInCli = exports.useCodeScanningConfigInCli = exports.isInTestMode = exports.getMlPoweredJsQueriesStatus = exports.getMlPoweredJsQueriesPack = exports.ML_POWERED_JS_QUERIES_PACK_NAME = exports.isGoodVersion = exports.delay = exports.bundleDb = exports.codeQlVersionAbove = exports.getCachedCodeQlVersion = exports.cacheCodeQlVersion = exports.isHTTPError = exports.UserError = exports.HTTPError = exports.getRequiredEnvParam = exports.enrichEnvironment = exports.initializeEnvironment = exports.EnvVar = exports.assertNever = exports.apiVersionInRange = exports.DisallowedAPIVersionReason = exports.checkGitHubVersionInRange = exports.getGitHubVersion = exports.GitHubVariant = exports.parseGitHubUrl = exports.getCodeQLDatabasePath = exports.getThreadsFlag = exports.getThreadsFlagValue = exports.getAddSnippetsFlag = exports.getMemoryFlag = exports.getMemoryFlagValue = exports.withTmpDir = exports.getToolNames = exports.getExtraOptionsEnvParam = exports.DID_AUTOBUILD_GO_ENV_VAR_NAME = exports.DEFAULT_DEBUG_DATABASE_NAME = exports.DEFAULT_DEBUG_ARTIFACT_NAME = exports.GITHUB_DOTCOM_URL = void 0;
const fs = __importStar(require("fs"));
const os = __importStar(require("os"));
const path = __importStar(require("path"));
@@ -40,7 +40,6 @@ const apiCompatibility = __importStar(require("./api-compatibility.json"));
const codeql_1 = require("./codeql");
const config_utils_1 = require("./config-utils");
const feature_flags_1 = require("./feature-flags");
const languages_1 = require("./languages");
const shared_environment_1 = require("./shared-environment");
/**
* Specifies bundle versions that are known to be broken
@@ -685,45 +684,6 @@ function isHostedRunner() {
process.env["RUNNER_TOOL_CACHE"]?.includes("hostedtoolcache"));
}
exports.isHostedRunner = isHostedRunner;
/**
*
* @param featuresEnablement The features enabled for the current run
* @param languagesInput Languages input from the workflow
* @param repository The owner/name of the repository
* @param logger A logger
* @returns A boolean indicating whether or not the toolcache should be bypassed and the latest codeql should be downloaded.
*/
async function shouldBypassToolcache(featuresEnablement, codeqlUrl, languagesInput, repository, logger) {
// An explicit codeql url is specified, that means the toolcache will not be used.
if (codeqlUrl) {
return true;
}
// Check if the toolcache is disabled for all languages
if (await featuresEnablement.getValue(feature_flags_1.Feature.BypassToolcacheEnabled)) {
return true;
}
// Check if the toolcache is disabled for kotlin and swift.
if (!(await featuresEnablement.getValue(feature_flags_1.Feature.BypassToolcacheKotlinSwiftEnabled))) {
return false;
}
// Now check to see if kotlin or swift is one of the languages being analyzed.
const { rawLanguages, autodetected } = await (0, config_utils_1.getRawLanguages)(languagesInput, repository, logger);
let bypass = rawLanguages.some((lang) => languages_1.KOTLIN_SWIFT_BYPASS.includes(lang));
if (bypass) {
logger.info(`Bypassing toolcache for kotlin or swift. Languages: ${rawLanguages}`);
}
else if (!autodetected && rawLanguages.includes(languages_1.Language.java)) {
// special case: java was explicitly specified, but there might be
// some kotlin in the repository, so we need to make a request for that.
const langsInRepo = await (0, config_utils_1.getLanguagesInRepo)(repository, logger);
if (langsInRepo.includes("kotlin")) {
logger.info(`Bypassing toolcache for kotlin.`);
bypass = true;
}
}
return bypass;
}
exports.shouldBypassToolcache = shouldBypassToolcache;
function parseMatrixInput(matrixInput) {
if (matrixInput === undefined || matrixInput === "null") {
return undefined;
+1 -1
View File
File diff suppressed because one or more lines are too long
-115
View File
@@ -33,9 +33,7 @@ const github = __importStar(require("@actions/github"));
const ava_1 = __importDefault(require("ava"));
const sinon = __importStar(require("sinon"));
const api = __importStar(require("./api-client"));
const feature_flags_1 = require("./feature-flags");
const logging_1 = require("./logging");
const repository_1 = require("./repository");
const testing_utils_1 = require("./testing-utils");
const util = __importStar(require("./util"));
(0, testing_utils_1.setupTests)(ava_1.default);
@@ -325,117 +323,4 @@ const shortTime = 10;
t.deepEqual(shortTaskTimedOut, false);
t.deepEqual(result, 99);
});
const mockRepositoryNwo = (0, repository_1.parseRepositoryNwo)("owner/repo");
// eslint-disable-next-line github/array-foreach
[
{
name: "disabled",
features: [],
hasCustomCodeQL: false,
languagesInput: undefined,
languagesInRepository: [],
expected: false,
expectedApiCall: false,
},
{
name: "disabled even though swift kotlin bypassed",
features: [feature_flags_1.Feature.BypassToolcacheKotlinSwiftEnabled],
hasCustomCodeQL: false,
languagesInput: undefined,
languagesInRepository: [],
expected: false,
expectedApiCall: true,
},
{
name: "disabled even though swift kotlin analyzed",
features: [],
hasCustomCodeQL: false,
languagesInput: " sWiFt , KoTlIn ",
languagesInRepository: [],
expected: false,
expectedApiCall: false,
},
{
name: "toolcache bypass all",
features: [feature_flags_1.Feature.BypassToolcacheEnabled],
hasCustomCodeQL: false,
languagesInput: undefined,
languagesInRepository: [],
expected: true,
expectedApiCall: false,
},
{
name: "custom CodeQL",
features: [],
hasCustomCodeQL: true,
languagesInput: undefined,
languagesInRepository: [],
expected: true,
expectedApiCall: false,
},
{
name: "bypass swift",
features: [feature_flags_1.Feature.BypassToolcacheKotlinSwiftEnabled],
hasCustomCodeQL: false,
languagesInput: " sWiFt ,other",
languagesInRepository: [],
expected: true,
expectedApiCall: false,
},
{
name: "bypass kotlin",
features: [feature_flags_1.Feature.BypassToolcacheKotlinSwiftEnabled],
hasCustomCodeQL: false,
languagesInput: "other, KoTlIn ",
languagesInRepository: [],
expected: true,
expectedApiCall: false,
},
{
name: "bypass kotlin language from repository",
features: [feature_flags_1.Feature.BypassToolcacheKotlinSwiftEnabled],
hasCustomCodeQL: false,
languagesInput: "",
languagesInRepository: ["KoTlIn", "other"],
expected: true,
expectedApiCall: true,
},
{
name: "bypass swift language from repository",
features: [feature_flags_1.Feature.BypassToolcacheKotlinSwiftEnabled],
hasCustomCodeQL: false,
languagesInput: "",
languagesInRepository: ["SwiFt", "other"],
expected: true,
expectedApiCall: true,
},
{
name: "bypass java from input if there is kotlin in repository",
features: [feature_flags_1.Feature.BypassToolcacheKotlinSwiftEnabled],
hasCustomCodeQL: false,
languagesInput: "java",
languagesInRepository: ["kotlin", "other"],
expected: true,
expectedApiCall: true,
},
{
name: "don't bypass java from input if there is no kotlin in repository",
features: [feature_flags_1.Feature.BypassToolcacheKotlinSwiftEnabled],
hasCustomCodeQL: false,
languagesInput: "java",
languagesInRepository: ["java", "other"],
expected: false,
expectedApiCall: true,
},
].forEach((args) => {
(0, ava_1.default)(`shouldBypassToolcache: ${args.name}`, async (t) => {
const mockRequest = (0, testing_utils_1.mockLanguagesInRepo)(args.languagesInRepository);
const mockLogger = (0, logging_1.getRunnerLogger)(true);
const featureEnablement = (0, testing_utils_1.createFeatures)(args.features);
const codeqlUrl = args.hasCustomCodeQL ? "custom-codeql-url" : undefined;
const actual = await util.shouldBypassToolcache(featureEnablement, codeqlUrl, args.languagesInput, mockRepositoryNwo, mockLogger);
t.deepEqual(actual, args.expected);
t.deepEqual(mockRequest.called, args.expectedApiCall);
});
});
//# sourceMappingURL=util.test.js.map
+1 -1
View File
File diff suppressed because one or more lines are too long
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "codeql",
"version": "2.2.2",
"version": "2.2.4",
"lockfileVersion": 3,
"requires": true,
"packages": {
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "codeql",
"version": "2.2.2",
"version": "2.2.4",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "codeql",
"version": "2.2.2",
"version": "2.2.4",
"license": "MIT",
"dependencies": {
"@actions/artifact": "^1.1.0",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "codeql",
"version": "2.2.2",
"version": "2.2.4",
"private": true,
"description": "CodeQL action",
"scripts": {
+105 -58
View File
@@ -87,10 +87,14 @@ test.beforeEach(() => {
function mockDownloadApi({
apiDetails = sampleApiDetails,
isPinned,
repo = "github/codeql-action",
platformSpecific = true,
tagName,
}: {
apiDetails?: GitHubApiDetails;
isPinned?: boolean;
repo?: string;
platformSpecific?: boolean;
tagName: string;
}): string {
const platform =
@@ -102,7 +106,9 @@ function mockDownloadApi({
const baseUrl = apiDetails?.url ?? "https://example.com";
const relativeUrl = apiDetails
? `/github/codeql-action/releases/download/${tagName}/codeql-bundle-${platform}.tar.gz`
? `/${repo}/releases/download/${tagName}/codeql-bundle${
platformSpecific ? `-${platform}` : ""
}.tar.gz`
: `/download/${tagName}/codeql-bundle.tar.gz`;
nock(baseUrl)
@@ -137,7 +143,6 @@ async function installIntoToolcache({
apiDetails,
tmpDir,
util.GitHubVariant.GHES,
false,
cliVersion !== undefined
? { cliVersion, tagName, variant: util.GitHubVariant.GHES }
: SAMPLE_DEFAULT_CLI_VERSION,
@@ -199,7 +204,6 @@ test("downloads and caches explicitly requested bundles that aren't in the toolc
sampleApiDetails,
tmpDir,
util.GitHubVariant.DOTCOM,
false,
SAMPLE_DEFAULT_CLI_VERSION,
getRunnerLogger(true),
false
@@ -233,7 +237,6 @@ test("downloads an explicitly requested bundle even if a different version is ca
sampleApiDetails,
tmpDir,
util.GitHubVariant.DOTCOM,
false,
SAMPLE_DEFAULT_CLI_VERSION,
getRunnerLogger(true),
false
@@ -284,7 +287,6 @@ for (const {
sampleApiDetails,
tmpDir,
util.GitHubVariant.DOTCOM,
false,
SAMPLE_DEFAULT_CLI_VERSION,
getRunnerLogger(true),
false
@@ -352,7 +354,6 @@ for (const { githubReleases, toolcacheVersion } of [
sampleApiDetails,
tmpDir,
util.GitHubVariant.DOTCOM,
false,
SAMPLE_DEFAULT_CLI_VERSION,
getRunnerLogger(true),
false
@@ -381,7 +382,6 @@ for (const variant of [util.GitHubVariant.GHAE, util.GitHubVariant.GHES]) {
sampleApiDetails,
tmpDir,
variant,
false,
{
cliVersion: defaults.cliVersion,
tagName: defaults.bundleVersion,
@@ -417,7 +417,6 @@ for (const variant of [util.GitHubVariant.GHAE, util.GitHubVariant.GHES]) {
sampleApiDetails,
tmpDir,
variant,
false,
{
cliVersion: defaults.cliVersion,
tagName: defaults.bundleVersion,
@@ -454,7 +453,6 @@ test('downloads bundle if "latest" tools specified but not cached', async (t) =>
sampleApiDetails,
tmpDir,
util.GitHubVariant.DOTCOM,
false,
SAMPLE_DEFAULT_CLI_VERSION,
getRunnerLogger(true),
false
@@ -468,69 +466,118 @@ test('downloads bundle if "latest" tools specified but not cached', async (t) =>
});
});
test("download codeql bundle from github ae endpoint", async (t) => {
for (const isBundleVersionInUrl of [true, false]) {
const inclusionString = isBundleVersionInUrl
? "includes"
: "does not include";
test(`download codeql bundle from github ae endpoint (URL ${inclusionString} bundle version)`, async (t) => {
await util.withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
const bundleAssetID = 10;
const platform =
process.platform === "win32"
? "win64"
: process.platform === "linux"
? "linux64"
: "osx64";
const codeQLBundleName = `codeql-bundle-${platform}.tar.gz`;
const eventualDownloadUrl = isBundleVersionInUrl
? `https://example.githubenterprise.com/github/codeql-action/releases/download/${defaults.bundleVersion}/${codeQLBundleName}`
: `https://example.githubenterprise.com/api/v3/repos/github/codeql-action/releases/assets/${bundleAssetID}`;
nock("https://example.githubenterprise.com")
.get(
`/api/v3/enterprise/code-scanning/codeql-bundle/find/${defaults.bundleVersion}`
)
.reply(200, {
assets: { [codeQLBundleName]: bundleAssetID },
});
nock("https://example.githubenterprise.com")
.get(
`/api/v3/enterprise/code-scanning/codeql-bundle/download/${bundleAssetID}`
)
.reply(200, {
url: eventualDownloadUrl,
});
nock("https://example.githubenterprise.com")
.get(
eventualDownloadUrl.replace(
"https://example.githubenterprise.com",
""
)
)
.replyWithFile(
200,
path.join(__dirname, `/../src/testdata/codeql-bundle-pinned.tar.gz`)
);
mockApiDetails(sampleGHAEApiDetails);
sinon.stub(actionsUtil, "isRunningLocalAction").returns(false);
process.env["GITHUB_ACTION_REPOSITORY"] = "github/codeql-action";
const result = await codeql.setupCodeQL(
undefined,
sampleGHAEApiDetails,
tmpDir,
util.GitHubVariant.GHAE,
{
cliVersion: defaults.cliVersion,
tagName: defaults.bundleVersion,
variant: util.GitHubVariant.GHAE,
},
getRunnerLogger(true),
false
);
t.is(result.toolsSource, ToolsSource.Download);
t.assert(Number.isInteger(result.toolsDownloadDurationMs));
const cachedVersions = toolcache.findAllVersions("CodeQL");
t.is(cachedVersions.length, 1);
});
});
}
test("bundle URL from another repo is cached as 0.0.0-bundleVersion", async (t) => {
await util.withTmpDir(async (tmpDir) => {
setupActionsVars(tmpDir, tmpDir);
const bundleAssetID = 10;
const platform =
process.platform === "win32"
? "win64"
: process.platform === "linux"
? "linux64"
: "osx64";
const codeQLBundleName = `codeql-bundle-${platform}.tar.gz`;
nock("https://example.githubenterprise.com")
.get(
`/api/v3/enterprise/code-scanning/codeql-bundle/find/${defaults.bundleVersion}`
)
.reply(200, {
assets: { [codeQLBundleName]: bundleAssetID },
});
nock("https://example.githubenterprise.com")
.get(
`/api/v3/enterprise/code-scanning/codeql-bundle/download/${bundleAssetID}`
)
.reply(200, {
url: `https://example.githubenterprise.com/github/codeql-action/releases/download/${defaults.bundleVersion}/${codeQLBundleName}`,
});
nock("https://example.githubenterprise.com")
.get(
`/github/codeql-action/releases/download/${defaults.bundleVersion}/${codeQLBundleName}`
)
.replyWithFile(
200,
path.join(__dirname, `/../src/testdata/codeql-bundle-pinned.tar.gz`)
);
mockApiDetails(sampleGHAEApiDetails);
sinon.stub(actionsUtil, "isRunningLocalAction").returns(false);
process.env["GITHUB_ACTION_REPOSITORY"] = "github/codeql-action";
mockApiDetails(sampleApiDetails);
sinon.stub(actionsUtil, "isRunningLocalAction").returns(true);
const releasesApiMock = mockReleaseApi({
assetNames: ["cli-version-2.12.2.txt"],
tagName: "codeql-bundle-20230203",
});
mockDownloadApi({
repo: "dsp-testing/codeql-cli-nightlies",
platformSpecific: false,
tagName: "codeql-bundle-20230203",
});
const result = await codeql.setupCodeQL(
undefined,
sampleGHAEApiDetails,
"https://github.com/dsp-testing/codeql-cli-nightlies/releases/download/codeql-bundle-20230203/codeql-bundle.tar.gz",
sampleApiDetails,
tmpDir,
util.GitHubVariant.GHAE,
false,
{
cliVersion: defaults.cliVersion,
tagName: defaults.bundleVersion,
variant: util.GitHubVariant.GHAE,
},
util.GitHubVariant.DOTCOM,
SAMPLE_DEFAULT_CLI_VERSION,
getRunnerLogger(true),
false
);
t.is(result.toolsVersion, "0.0.0-20230203");
t.is(result.toolsSource, ToolsSource.Download);
t.assert(Number.isInteger(result.toolsDownloadDurationMs));
t.true(Number.isInteger(result.toolsDownloadDurationMs));
const cachedVersions = toolcache.findAllVersions("CodeQL");
t.is(cachedVersions.length, 1);
t.is(cachedVersions[0], "0.0.0-20230203");
t.false(releasesApiMock.isDone());
});
});
+5 -3
View File
@@ -278,6 +278,11 @@ export const CODEQL_VERSION_ML_POWERED_QUERIES_WINDOWS = "2.9.0";
*/
export const CODEQL_VERSION_BETTER_RESOLVE_LANGUAGES = "2.10.3";
/**
* Versions 2.11.1+ of the CodeQL Bundle include a `security-experimental` built-in query suite for each language.
*/
export const CODEQL_VERSION_SECURITY_EXPERIMENTAL_SUITE = "2.12.1";
/**
* Set up CodeQL CLI access.
*
@@ -285,7 +290,6 @@ export const CODEQL_VERSION_BETTER_RESOLVE_LANGUAGES = "2.10.3";
* @param apiDetails
* @param tempDir
* @param variant
* @param bypassToolcache
* @param defaultCliVersion
* @param logger
* @param checkVersion Whether to check that CodeQL CLI meets the minimum
@@ -297,7 +301,6 @@ export async function setupCodeQL(
apiDetails: api.GitHubApiDetails,
tempDir: string,
variant: util.GitHubVariant,
bypassToolcache: boolean,
defaultCliVersion: CodeQLDefaultVersionInfo,
logger: Logger,
checkVersion: boolean
@@ -314,7 +317,6 @@ export async function setupCodeQL(
apiDetails,
tempDir,
variant,
bypassToolcache,
defaultCliVersion,
logger
);
+11 -2
View File
@@ -1993,7 +1993,7 @@ test(
process.platform === "win32" ? undefined : "~0.1.0"
);
// Test that ML-powered queries aren't run when the user hasn't specified that we should run the
// `security-extended` or `security-and-quality` query suite.
// `security-extended`, `security-and-quality`, or `security-experimental` query suite.
test(mlPoweredQueriesMacro, "2.7.5", true, undefined, undefined, undefined);
// Test that ML-powered queries are run on non-Windows platforms running `security-extended` on
// versions of the CodeQL CLI prior to 2.9.0.
@@ -2074,7 +2074,6 @@ test(
"security-extended",
"~0.4.0"
);
// Test that ML-powered queries are run on all platforms running `security-and-quality` on CodeQL
// CLI 2.11.3+.
test(
@@ -2085,6 +2084,16 @@ test(
"security-and-quality",
"~0.4.0"
);
// Test that ML-powered queries are run on all platforms running `security-experimental` on CodeQL
// CLI 2.12.1+.
test(
mlPoweredQueriesMacro,
"2.12.1",
true,
undefined,
"security-experimental",
"~0.4.0"
);
const calculateAugmentationMacro = test.macro({
exec: async (
+24 -3
View File
@@ -10,6 +10,7 @@ import {
CodeQL,
CODEQL_VERSION_GHES_PACK_DOWNLOAD,
CODEQL_VERSION_ML_POWERED_QUERIES_WINDOWS,
CODEQL_VERSION_SECURITY_EXPERIMENTAL_SUITE,
ResolveQueriesOutput,
} from "./codeql";
import * as externalQueries from "./external-queries";
@@ -380,7 +381,11 @@ async function addDefaultQueries(
}
// The set of acceptable values for built-in suites from the codeql bundle
const builtinSuites = ["security-extended", "security-and-quality"] as const;
const builtinSuites = [
"security-experimental",
"security-extended",
"security-and-quality",
] as const;
/**
* Determine the set of queries associated with suiteName's suites and add them to resultMap.
@@ -401,6 +406,19 @@ async function addBuiltinSuiteQueries(
if (!found) {
throw new Error(getQueryUsesInvalid(configFile, suiteName));
}
if (
suiteName === "security-experimental" &&
!(await codeQlVersionAbove(
codeQL,
CODEQL_VERSION_SECURITY_EXPERIMENTAL_SUITE
))
) {
throw new Error(
`The 'security-experimental' suite is not supported on CodeQL CLI versions earlier than
${CODEQL_VERSION_SECURITY_EXPERIMENTAL_SUITE}. Please upgrade to CodeQL CLI version
${CODEQL_VERSION_SECURITY_EXPERIMENTAL_SUITE} or later.`
);
}
// If we're running the JavaScript security-extended analysis (or a superset of it), the repo is
// opted into the ML-powered queries beta, and a user hasn't already added the ML-powered query
@@ -413,7 +431,9 @@ async function addBuiltinSuiteQueries(
CODEQL_VERSION_ML_POWERED_QUERIES_WINDOWS
))) &&
languages.includes("javascript") &&
(found === "security-extended" || found === "security-and-quality") &&
(found === "security-experimental" ||
found === "security-extended" ||
found === "security-and-quality") &&
!packs.javascript?.some(isMlPoweredJsQueriesPack) &&
(await featureEnablement.getValue(Feature.MlPoweredQueriesEnabled, codeQL))
) {
@@ -1630,7 +1650,8 @@ export function parsePacks(
* Without a '+', an input value will override the corresponding value in the config file.
*
* @param inputValue The input value to process.
* @returns true if the input value should replace the corresponding value in the config file, false if it should be appended.
* @returns true if the input value should replace the corresponding value in the config file,
* false if it should be appended.
*/
function shouldCombine(inputValue?: string): boolean {
return !!inputValue?.trim().startsWith("+");
+4 -4
View File
@@ -1,6 +1,6 @@
{
"bundleVersion": "codeql-bundle-20230120",
"cliVersion": "2.12.1",
"priorBundleVersion": "codeql-bundle-20230105",
"priorCliVersion": "2.12.0"
"bundleVersion": "codeql-bundle-20230207",
"cliVersion": "2.12.2",
"priorBundleVersion": "codeql-bundle-20230120",
"priorCliVersion": "2.12.1"
}
+15 -24
View File
@@ -34,8 +34,6 @@ export interface FeatureEnablement {
}
export enum Feature {
BypassToolcacheEnabled = "bypass_toolcache_enabled",
BypassToolcacheKotlinSwiftEnabled = "bypass_toolcache_kotlin_swift_enabled",
CliConfigFileEnabled = "cli_config_file_enabled",
DisableKotlinAnalysisEnabled = "disable_kotlin_analysis_enabled",
MlPoweredQueriesEnabled = "ml_powered_queries_enabled",
@@ -47,18 +45,6 @@ export const featureConfig: Record<
Feature,
{ envVar: string; minimumVersion: string | undefined }
> = {
[Feature.BypassToolcacheEnabled]: {
envVar: "CODEQL_BYPASS_TOOLCACHE",
// Cannot specify a minimum version because this flag is checked before we have
// access to the CodeQL instance.
minimumVersion: undefined,
},
[Feature.BypassToolcacheKotlinSwiftEnabled]: {
envVar: "CODEQL_BYPASS_TOOLCACHE_KOTLIN_SWIFT",
// Cannot specify a minimum version because this flag is checked before we have
// access to the CodeQL instance.
minimumVersion: undefined,
},
[Feature.DisableKotlinAnalysisEnabled]: {
envVar: "CODEQL_DISABLE_KOTLIN_ANALYSIS",
minimumVersion: undefined,
@@ -138,11 +124,6 @@ export class Features implements FeatureEnablement {
);
}
// Bypassing the toolcache is disabled in test mode.
if (feature === Feature.BypassToolcacheEnabled && util.isInTestMode()) {
return false;
}
const envVar = (
process.env[featureConfig[feature].envVar] || ""
).toLocaleLowerCase();
@@ -172,13 +153,17 @@ export class Features implements FeatureEnablement {
class GitHubFeatureFlags implements FeatureEnablement {
private cachedApiResponse: GitHubFeatureFlagsApiResponse | undefined;
// We cache whether the feature flags were accessed or not in order to accurately report whether flags were
// incorrectly configured vs. inaccessible in our telemetry.
private hasAccessedRemoteFeatureFlags: boolean;
constructor(
private readonly gitHubVersion: util.GitHubVersion,
private readonly repositoryNwo: RepositoryNwo,
private readonly featureFlagsFile: string,
private readonly logger: Logger
) {
/**/
this.hasAccessedRemoteFeatureFlags = false; // Not accessed by default.
}
private getCliVersionFromFeatureFlag(f: string): string | undefined {
@@ -211,7 +196,9 @@ class GitHubFeatureFlags implements FeatureEnablement {
const defaultDotComCliVersion = await this.getDefaultDotcomCliVersion();
return {
cliVersion: defaultDotComCliVersion.version,
toolsFeatureFlagsValid: defaultDotComCliVersion.toolsFeatureFlagsValid,
toolsFeatureFlagsValid: this.hasAccessedRemoteFeatureFlags
? defaultDotComCliVersion.toolsFeatureFlagsValid
: undefined,
variant,
};
}
@@ -224,7 +211,7 @@ class GitHubFeatureFlags implements FeatureEnablement {
async getDefaultDotcomCliVersion(): Promise<{
version: string;
toolsFeatureFlagsValid: boolean;
toolsFeatureFlagsValid: boolean | undefined;
}> {
const response = await this.getAllFeatures();
@@ -252,7 +239,9 @@ class GitHubFeatureFlags implements FeatureEnablement {
);
return {
version: defaults.cliVersion,
toolsFeatureFlagsValid: false,
toolsFeatureFlagsValid: this.hasAccessedRemoteFeatureFlags
? false
: undefined,
};
}
@@ -350,6 +339,7 @@ class GitHubFeatureFlags implements FeatureEnablement {
this.logger.debug(
"Not running against github.com. Disabling all toggleable features."
);
this.hasAccessedRemoteFeatureFlags = false;
return {};
}
try {
@@ -365,6 +355,7 @@ class GitHubFeatureFlags implements FeatureEnablement {
"Loaded the following default values for the feature flags from the Code Scanning API: " +
`${JSON.stringify(remoteFlags)}`
);
this.hasAccessedRemoteFeatureFlags = true;
return remoteFlags;
} catch (e) {
if (util.isHTTPError(e) && e.status === 403) {
@@ -374,6 +365,7 @@ class GitHubFeatureFlags implements FeatureEnablement {
"This could be because the Action is running on a pull request from a fork. If not, " +
`please ensure the Action has the 'security-events: write' permission. Details: ${e}`
);
this.hasAccessedRemoteFeatureFlags = false;
return {};
} else {
// Some features, such as `ml_powered_queries_enabled` affect the produced alerts.
@@ -385,6 +377,5 @@ class GitHubFeatureFlags implements FeatureEnablement {
);
}
}
return {};
}
}
+3 -9
View File
@@ -43,7 +43,6 @@ import {
GitHubVariant,
initializeEnvironment,
isHostedRunner,
shouldBypassToolcache,
} from "./util";
import { validateWorkflow } from "./workflow";
@@ -87,7 +86,9 @@ interface InitWithConfigStatusReport extends InitStatusReport {
interface InitToolsDownloadFields {
/** Time taken to download the bundle, in milliseconds. */
tools_download_duration_ms?: number;
/** Whether the relevant tools dotcom feature flags have been misconfigured. Only populated if we attempt to determine the default version based on the dotcom feature flags. */
/**
* Whether the relevant tools dotcom feature flags have been misconfigured.
* Only populated if we attempt to determine the default version based on the dotcom feature flags. */
tools_feature_flags_valid?: boolean;
}
@@ -236,13 +237,6 @@ async function run() {
apiDetails,
getTemporaryDirectory(),
gitHubVersion.type,
await shouldBypassToolcache(
features,
getOptionalInput("tools"),
getOptionalInput("languages"),
repositoryNwo,
logger
),
codeQLDefaultVersionInfo,
logger
);
-2
View File
@@ -27,7 +27,6 @@ export async function initCodeQL(
apiDetails: GitHubApiDetails,
tempDir: string,
variant: util.GitHubVariant,
bypassToolcache: boolean,
defaultCliVersion: CodeQLDefaultVersionInfo,
logger: Logger
): Promise<{
@@ -43,7 +42,6 @@ export async function initCodeQL(
apiDetails,
tempDir,
variant,
bypassToolcache,
defaultCliVersion,
logger,
true
-2
View File
@@ -21,8 +21,6 @@ export const LANGUAGE_ALIASES: { [lang: string]: Language } = {
export type LanguageOrAlias = Language | keyof typeof LANGUAGE_ALIASES;
export const KOTLIN_SWIFT_BYPASS = ["kotlin", "swift"];
export function resolveAlias(lang: LanguageOrAlias): Language {
return LANGUAGE_ALIASES[lang] || lang;
}
+221 -145
View File
@@ -241,24 +241,36 @@ async function getCodeQLBundleDownloadURL(
return `https://github.com/${CODEQL_DEFAULT_ACTION_REPOSITORY}/releases/download/${tagName}/${codeQLBundleName}`;
}
function getBundleVersionFromTagName(tagName: string): string {
function tryGetBundleVersionFromTagName(
tagName: string,
logger: Logger
): string | undefined {
const match = tagName.match(/^codeql-bundle-(.*)$/);
if (match === null || match.length < 2) {
throw new Error(
`Malformed bundle tag name: ${tagName}. Bundle version could not be inferred`
);
logger.debug(`Could not determine bundle version from tag ${tagName}.`);
return undefined;
}
return match[1];
}
export function getBundleVersionFromUrl(url: string): string {
function tryGetTagNameFromUrl(url: string, logger: Logger): string | undefined {
const match = url.match(/\/(codeql-bundle-.*)\//);
if (match === null || match.length < 2) {
throw new Error(
`Malformed tools url: ${url}. Bundle version could not be inferred`
);
logger.debug(`Could not determine tag name for URL ${url}.`);
return undefined;
}
return getBundleVersionFromTagName(match[1]);
return match[1];
}
export function tryGetBundleVersionFromUrl(
url: string,
logger: Logger
): string | undefined {
const tagName = tryGetTagNameFromUrl(url, logger);
if (tagName === undefined) {
return undefined;
}
return tryGetBundleVersionFromTagName(tagName, logger);
}
export function convertToSemVer(version: string, logger: Logger): string {
@@ -291,6 +303,8 @@ type CodeQLToolsSource =
toolsVersion: string;
}
| {
/** Bundle version of the tools, if known. */
bundleVersion?: string;
/** CLI version of the tools, if known. */
cliVersion?: string;
codeqlURL: string;
@@ -299,22 +313,11 @@ type CodeQLToolsSource =
toolsVersion: string;
};
async function getOrFindBundleTagName(
version: CodeQLDefaultVersionInfo,
logger: Logger
): Promise<string> {
if (version.variant === util.GitHubVariant.DOTCOM) {
return await findCodeQLBundleTagDotcomOnly(version.cliVersion, logger);
} else {
return version.tagName;
}
}
/**
* Look for a version of the CodeQL tools in the cache which could override the requested CLI version.
*/
async function findOverridingToolsInCache(
requestedCliVersion: string,
humanReadableVersion: string,
logger: Logger
): Promise<CodeQLToolsSource | undefined> {
const candidates = toolcache
@@ -329,7 +332,7 @@ async function findOverridingToolsInCache(
if (candidates.length === 1) {
const candidate = candidates[0];
logger.debug(
`CodeQL tools version ${candidate.version} in toolcache overriding version ${requestedCliVersion}.`
`CodeQL tools version ${candidate.version} in toolcache overriding version ${humanReadableVersion}.`
);
return {
codeqlFolder: candidate.folder,
@@ -351,7 +354,6 @@ async function findOverridingToolsInCache(
export async function getCodeQLSource(
toolsInput: string | undefined,
bypassToolcache: boolean,
defaultCliVersion: CodeQLDefaultVersionInfo,
apiDetails: api.GitHubApiDetails,
variant: util.GitHubVariant,
@@ -365,76 +367,73 @@ export async function getCodeQLSource(
};
}
const forceLatestReason =
// We use the special value of 'latest' to prioritize the version in the
// defaults over any pinned cached version.
toolsInput === "latest"
? '"tools: latest" was requested'
: // If the user hasn't requested a particular CodeQL version, then bypass
// the toolcache when the appropriate feature is enabled. This
// allows us to quickly rollback a broken bundle that has made its way
// into the toolcache.
toolsInput === undefined && bypassToolcache
? "a specific version of the CodeQL tools was not requested and the bypass toolcache feature is enabled"
: undefined;
const forceLatest = forceLatestReason !== undefined;
if (forceLatest) {
logger.debug(
`Forcing the latest version of the CodeQL tools since ${forceLatestReason}.`
/**
* Whether the tools shipped with the Action, i.e. those in `defaults.json`, have been forced.
*
* We use the special value of 'latest' to prioritize the version in `defaults.json` over the
* version specified by the feature flags on Dotcom and over any pinned cached version on
* Enterprise Server.
*/
const forceShippedTools = toolsInput === "latest";
if (forceShippedTools) {
logger.info(
"Overriding the version of the CodeQL tools by the version shipped with the Action since " +
`"tools: latest" was requested.`
);
}
/** CLI version number, for example 2.12.1. */
let cliVersion: string | undefined;
/** Tag name of the CodeQL bundle, for example `codeql-bundle-20230120`. */
let tagName: string | undefined;
/**
* The requested version is:
*
* 1. The one in `defaults.json`, if forceLatest is true.
* 2. The version specified by the tools input URL, if one was provided.
* 3. The default CLI version, otherwise.
* We include a `variant` property to let us verify using the type system that
* `tagName` is only undefined when the variant is Dotcom. This lets us ensure
* that we can always compute `tagName`, either by using the existing tag name
* on enterprise instances, or calling `findCodeQLBundleTagDotcomOnly` on
* Dotcom.
* URL of the CodeQL bundle.
*
* This does not always include a tag name.
*/
const requestedVersion = forceLatest
? // case 1
{
cliVersion: defaults.cliVersion,
syntheticCliVersion: defaults.cliVersion,
tagName: defaults.bundleVersion,
variant,
}
: toolsInput !== undefined
? // case 2
{
syntheticCliVersion: convertToSemVer(
getBundleVersionFromUrl(toolsInput),
logger
),
tagName: `codeql-bundle-${getBundleVersionFromUrl(toolsInput)}`,
url: toolsInput,
variant,
}
: // case 3
{
...defaultCliVersion,
syntheticCliVersion: defaultCliVersion.cliVersion,
};
let url: string | undefined;
// If we find the specified version, we always use that.
let codeqlFolder = toolcache.find(
"CodeQL",
requestedVersion.syntheticCliVersion
if (forceShippedTools) {
cliVersion = defaults.cliVersion;
tagName = defaults.bundleVersion;
} else if (toolsInput !== undefined) {
// If a tools URL was provided, then use that.
tagName = tryGetTagNameFromUrl(toolsInput, logger);
url = toolsInput;
} else {
// Otherwise, use the default CLI version passed in.
cliVersion = defaultCliVersion.cliVersion;
tagName = defaultCliVersion["tagName"];
}
const bundleVersion =
tagName && tryGetBundleVersionFromTagName(tagName, logger);
const humanReadableVersion =
cliVersion ??
(bundleVersion && convertToSemVer(bundleVersion, logger)) ??
tagName ??
url ??
"unknown";
logger.debug(
"Attempting to obtain CodeQL tools. " +
`CLI version: ${cliVersion ?? "unknown"}, ` +
`bundle tag name: ${tagName ?? "unknown"}, ` +
`URL: ${url ?? "unspecified"}.`
);
let tagName: string | undefined = requestedVersion["tagName"];
if (!codeqlFolder) {
logger.debug(
"Didn't find a version of the CodeQL tools in the toolcache with a version number " +
`exactly matching ${requestedVersion.syntheticCliVersion}.`
);
if (requestedVersion.cliVersion) {
let codeqlFolder;
if (cliVersion) {
// If we find the specified CLI version, we always use that.
codeqlFolder = toolcache.find("CodeQL", cliVersion);
// Fall back to matching `x.y.z-<tagName>`.
if (!codeqlFolder) {
logger.debug(
"Didn't find a version of the CodeQL tools in the toolcache with a version number " +
`exactly matching ${cliVersion}.`
);
const allVersions = toolcache.findAllVersions("CodeQL");
logger.debug(
`Found the following versions of the CodeQL tools in the toolcache: ${JSON.stringify(
@@ -444,55 +443,82 @@ export async function getCodeQLSource(
// If there is exactly one version of the CodeQL tools in the toolcache, and that version is
// the form `x.y.z-<tagName>`, then use it.
const candidateVersions = allVersions.filter((version) =>
version.startsWith(`${requestedVersion.cliVersion}-`)
version.startsWith(`${cliVersion}-`)
);
if (candidateVersions.length === 1) {
logger.debug("Exactly one candidate version found, using that.");
codeqlFolder = toolcache.find("CodeQL", candidateVersions[0]);
} else {
logger.debug(
"Did not find exactly one version of the CodeQL tools starting with the requested version."
`Exactly one version of the CodeQL tools starting with ${cliVersion} found in the ` +
"toolcache, using that."
);
codeqlFolder = toolcache.find("CodeQL", candidateVersions[0]);
} else if (candidateVersions.length === 0) {
logger.debug(
`Didn't find any versions of the CodeQL tools starting with ${cliVersion} ` +
`in the toolcache. Trying next fallback method.`
);
} else {
logger.warning(
`Found ${candidateVersions.length} versions of the CodeQL tools starting with ` +
`${cliVersion} in the toolcache, but at most one was expected.`
);
logger.debug("Trying next fallback method.");
}
}
}
if (!codeqlFolder && requestedVersion.cliVersion) {
// Fall back to accepting a `0.0.0-<bundleVersion>` version if we didn't find the
// `x.y.z` version. This is to support old versions of the toolcache.
//
// If we are on Dotcom, we will make an HTTP request to the Releases API here
// to find the tag name for the requested version.
tagName =
tagName || (await getOrFindBundleTagName(requestedVersion, logger));
const fallbackVersion = convertToSemVer(
getBundleVersionFromTagName(tagName),
logger
// Fall back to matching `0.0.0-<bundleVersion>`.
if (!codeqlFolder && (cliVersion || tagName)) {
if (cliVersion || tagName) {
const fallbackVersion = await tryGetFallbackToolcacheVersion(
cliVersion,
tagName,
variant,
logger
);
if (fallbackVersion) {
codeqlFolder = toolcache.find("CodeQL", fallbackVersion);
} else {
logger.debug(
"Could not determine a fallback toolcache version number for CodeQL tools version " +
`${humanReadableVersion}.`
);
}
} else {
logger.debug(
"Both the CLI version and the bundle version are unknown, so we will not be able to find " +
"the requested version of the CodeQL tools in the toolcache."
);
}
}
if (codeqlFolder) {
logger.info(
`Found CodeQL tools version ${humanReadableVersion} in the toolcache.`
);
logger.debug(
`Computed a fallback toolcache version number of ${fallbackVersion} for CodeQL tools version ` +
`${requestedVersion.cliVersion}.`
} else {
logger.info(
`Did not find CodeQL tools version ${humanReadableVersion} in the toolcache.`
);
codeqlFolder = toolcache.find("CodeQL", fallbackVersion);
}
if (codeqlFolder) {
return {
codeqlFolder,
sourceType: "toolcache",
toolsVersion: requestedVersion.syntheticCliVersion,
toolsVersion: cliVersion ?? humanReadableVersion,
};
}
logger.debug(
`Did not find CodeQL tools version ${requestedVersion.syntheticCliVersion} in the toolcache.`
);
// If we don't find the requested version on Enterprise, we may allow a
// different version to save download time if the version hasn't been
// specified explicitly (in which case we always honor it).
if (variant !== util.GitHubVariant.DOTCOM && !forceLatest && !toolsInput) {
if (
variant !== util.GitHubVariant.DOTCOM &&
!forceShippedTools &&
!toolsInput
) {
const result = await findOverridingToolsInCache(
requestedVersion.syntheticCliVersion,
humanReadableVersion,
logger
);
if (result !== undefined) {
@@ -500,30 +526,66 @@ export async function getCodeQLSource(
}
}
if (!url) {
if (!tagName && cliVersion && variant === util.GitHubVariant.DOTCOM) {
tagName = await findCodeQLBundleTagDotcomOnly(cliVersion, logger);
} else if (!tagName) {
throw new Error(
`Could not obtain the requested version (${humanReadableVersion}) of the CodeQL tools ` +
"since we could not compute the tag name."
);
}
url = await getCodeQLBundleDownloadURL(
tagName,
apiDetails,
variant,
logger
);
}
return {
cliVersion: requestedVersion.cliVersion || undefined,
codeqlURL:
requestedVersion["url"] ||
(await getCodeQLBundleDownloadURL(
tagName ||
// The check on `requestedVersion.tagName` is redundant but lets us
// use the property that if we don't know `requestedVersion.tagName`,
// then we must know `requestedVersion.cliVersion`. This property is
// required by the type of `getOrFindBundleTagName`.
(requestedVersion.tagName !== undefined
? requestedVersion.tagName
: await getOrFindBundleTagName(requestedVersion, logger)),
apiDetails,
variant,
logger
)),
bundleVersion: tagName && tryGetBundleVersionFromTagName(tagName, logger),
cliVersion,
codeqlURL: url,
sourceType: "download",
toolsVersion: requestedVersion.syntheticCliVersion,
toolsVersion: cliVersion ?? humanReadableVersion,
};
}
/**
* Gets a fallback version number to use when looking for CodeQL in the toolcache if we didn't find
* the `x.y.z` version. This is to support old versions of the toolcache.
*/
export async function tryGetFallbackToolcacheVersion(
cliVersion: string | undefined,
tagName: string | undefined,
variant: util.GitHubVariant,
logger: Logger
): Promise<string | undefined> {
//
// If we are on Dotcom, we will make an HTTP request to the Releases API here
// to find the tag name for the requested version.
if (cliVersion && !tagName && variant === util.GitHubVariant.DOTCOM) {
tagName = await findCodeQLBundleTagDotcomOnly(cliVersion, logger);
}
if (!tagName) {
return undefined;
}
const bundleVersion = tryGetBundleVersionFromTagName(tagName, logger);
if (!bundleVersion) {
return undefined;
}
const fallbackVersion = convertToSemVer(bundleVersion, logger);
logger.debug(
`Computed a fallback toolcache version number of ${fallbackVersion} for CodeQL version ` +
`${cliVersion ?? tagName}.`
);
return fallbackVersion;
}
export async function downloadCodeQL(
codeqlURL: string,
maybeBundleVersion: string | undefined,
maybeCliVersion: string | undefined,
apiDetails: api.GitHubApiDetails,
variant: util.GitHubVariant,
@@ -577,16 +639,33 @@ export async function downloadCodeQL(
const codeqlExtracted = await toolcache.extractTar(codeqlPath);
const bundleVersion = getBundleVersionFromUrl(codeqlURL);
const bundleVersion =
maybeBundleVersion ?? tryGetBundleVersionFromUrl(codeqlURL, logger);
if (bundleVersion === undefined) {
logger.debug(
"Could not cache CodeQL tools because we could not determine the bundle version from the " +
`URL ${codeqlURL}.`
);
return {
toolsVersion: maybeCliVersion ?? "unknown",
codeqlFolder: codeqlExtracted,
toolsDownloadDurationMs,
};
}
// Try to compute the CLI version for this bundle
const cliVersion: string | undefined =
maybeCliVersion ||
(variant === util.GitHubVariant.DOTCOM &&
(await tryFindCliVersionDotcomOnly(
`codeql-bundle-${bundleVersion}`,
logger
))) ||
undefined;
if (
maybeCliVersion === undefined &&
variant === util.GitHubVariant.DOTCOM &&
codeqlURL.includes(`/${CODEQL_DEFAULT_ACTION_REPOSITORY}/`)
) {
maybeCliVersion = await tryFindCliVersionDotcomOnly(
`codeql-bundle-${bundleVersion}`,
logger
);
}
// Include both the CLI version and the bundle version in the toolcache version number. That way
// if the user requests the same URL again, we can get it from the cache without having to call
// any of the Releases API.
@@ -596,12 +675,11 @@ export async function downloadCodeQL(
// CLI release. In principle, it should be enough to just check that the CLI version isn't a
// pre-release, but the version numbers of CodeQL nightlies have the format `x.y.z+<timestamp>`,
// and we don't want these nightlies to override stable CLI versions in the toolcache.
const toolcacheVersion =
cliVersion && cliVersion.match(/^[0-9]+\.[0-9]+\.[0-9]+$/)
? `${cliVersion}-${bundleVersion}`
: convertToSemVer(bundleVersion, logger);
const toolcacheVersion = maybeCliVersion?.match(/^[0-9]+\.[0-9]+\.[0-9]+$/)
? `${maybeCliVersion}-${bundleVersion}`
: convertToSemVer(bundleVersion, logger);
return {
toolsVersion: cliVersion || toolcacheVersion,
toolsVersion: maybeCliVersion ?? toolcacheVersion,
codeqlFolder: await toolcache.cacheDir(
codeqlExtracted,
"CodeQL",
@@ -628,7 +706,6 @@ export function getCodeQLURLVersion(url: string): string {
* @param apiDetails
* @param tempDir
* @param variant
* @param bypassToolcache
* @param defaultCliVersion
* @param logger
* @param checkVersion Whether to check that CodeQL CLI meets the minimum
@@ -640,7 +717,6 @@ export async function setupCodeQLBundle(
apiDetails: api.GitHubApiDetails,
tempDir: string,
variant: util.GitHubVariant,
bypassToolcache: boolean,
defaultCliVersion: CodeQLDefaultVersionInfo,
logger: Logger
): Promise<{
@@ -651,7 +727,6 @@ export async function setupCodeQLBundle(
}> {
const source = await getCodeQLSource(
toolsInput,
bypassToolcache,
defaultCliVersion,
apiDetails,
variant,
@@ -675,6 +750,7 @@ export async function setupCodeQLBundle(
case "download": {
const result = await downloadCodeQL(
source.codeqlURL,
source.bundleVersion,
source.cliVersion,
apiDetails,
variant,
+2 -1
View File
@@ -416,7 +416,8 @@ export async function waitForProcessing(
statusCheckingStarted + STATUS_CHECK_TIMEOUT_MILLISECONDS
) {
// If the analysis hasn't finished processing in the allotted time, we continue anyway rather than failing.
// It's possible the analysis will eventually finish processing, but it's not worth spending more Actions time waiting.
// It's possible the analysis will eventually finish processing, but it's not worth spending more
// Actions time waiting.
logger.warning(
"Timed out waiting for analysis to finish processing. Continuing."
);
+1 -127
View File
@@ -8,14 +8,8 @@ import * as sinon from "sinon";
import * as api from "./api-client";
import { Config } from "./config-utils";
import { Feature } from "./feature-flags";
import { getRunnerLogger } from "./logging";
import { parseRepositoryNwo } from "./repository";
import {
createFeatures,
mockLanguagesInRepo,
setupTests,
} from "./testing-utils";
import { setupTests } from "./testing-utils";
import * as util from "./util";
setupTests(test);
@@ -398,123 +392,3 @@ test("withTimeout doesn't call callback if promise resolves", async (t) => {
t.deepEqual(shortTaskTimedOut, false);
t.deepEqual(result, 99);
});
const mockRepositoryNwo = parseRepositoryNwo("owner/repo");
// eslint-disable-next-line github/array-foreach
[
{
name: "disabled",
features: [],
hasCustomCodeQL: false,
languagesInput: undefined,
languagesInRepository: [],
expected: false,
expectedApiCall: false,
},
{
name: "disabled even though swift kotlin bypassed",
features: [Feature.BypassToolcacheKotlinSwiftEnabled],
hasCustomCodeQL: false,
languagesInput: undefined,
languagesInRepository: [],
expected: false,
expectedApiCall: true,
},
{
name: "disabled even though swift kotlin analyzed",
features: [],
hasCustomCodeQL: false,
languagesInput: " sWiFt , KoTlIn ",
languagesInRepository: [],
expected: false,
expectedApiCall: false,
},
{
name: "toolcache bypass all",
features: [Feature.BypassToolcacheEnabled],
hasCustomCodeQL: false,
languagesInput: undefined,
languagesInRepository: [],
expected: true,
expectedApiCall: false,
},
{
name: "custom CodeQL",
features: [],
hasCustomCodeQL: true,
languagesInput: undefined,
languagesInRepository: [],
expected: true,
expectedApiCall: false,
},
{
name: "bypass swift",
features: [Feature.BypassToolcacheKotlinSwiftEnabled],
hasCustomCodeQL: false,
languagesInput: " sWiFt ,other",
languagesInRepository: [],
expected: true,
expectedApiCall: false,
},
{
name: "bypass kotlin",
features: [Feature.BypassToolcacheKotlinSwiftEnabled],
hasCustomCodeQL: false,
languagesInput: "other, KoTlIn ",
languagesInRepository: [],
expected: true,
expectedApiCall: false,
},
{
name: "bypass kotlin language from repository",
features: [Feature.BypassToolcacheKotlinSwiftEnabled],
hasCustomCodeQL: false,
languagesInput: "",
languagesInRepository: ["KoTlIn", "other"],
expected: true,
expectedApiCall: true,
},
{
name: "bypass swift language from repository",
features: [Feature.BypassToolcacheKotlinSwiftEnabled],
hasCustomCodeQL: false,
languagesInput: "",
languagesInRepository: ["SwiFt", "other"],
expected: true,
expectedApiCall: true,
},
{
name: "bypass java from input if there is kotlin in repository",
features: [Feature.BypassToolcacheKotlinSwiftEnabled],
hasCustomCodeQL: false,
languagesInput: "java",
languagesInRepository: ["kotlin", "other"],
expected: true,
expectedApiCall: true,
},
{
name: "don't bypass java from input if there is no kotlin in repository",
features: [Feature.BypassToolcacheKotlinSwiftEnabled],
hasCustomCodeQL: false,
languagesInput: "java",
languagesInRepository: ["java", "other"],
expected: false,
expectedApiCall: true,
},
].forEach((args) => {
test(`shouldBypassToolcache: ${args.name}`, async (t) => {
const mockRequest = mockLanguagesInRepo(args.languagesInRepository);
const mockLogger = getRunnerLogger(true);
const featureEnablement = createFeatures(args.features);
const codeqlUrl = args.hasCustomCodeQL ? "custom-codeql-url" : undefined;
const actual = await util.shouldBypassToolcache(
featureEnablement,
codeqlUrl,
args.languagesInput,
mockRepositoryNwo,
mockLogger
);
t.deepEqual(actual, args.expected);
t.deepEqual(mockRequest.called, args.expectedApiCall);
});
});
+1 -61
View File
@@ -13,15 +13,12 @@ import * as apiCompatibility from "./api-compatibility.json";
import { CodeQL, CODEQL_VERSION_NEW_TRACING } from "./codeql";
import {
Config,
getLanguagesInRepo,
getRawLanguages,
parsePacksSpecification,
prettyPrintPack,
} from "./config-utils";
import { Feature, FeatureEnablement } from "./feature-flags";
import { KOTLIN_SWIFT_BYPASS, Language } from "./languages";
import { Language } from "./languages";
import { Logger } from "./logging";
import { RepositoryNwo } from "./repository";
import { CODEQL_ACTION_TEST_MODE } from "./shared-environment";
/**
@@ -802,63 +799,6 @@ export function isHostedRunner() {
);
}
/**
*
* @param featuresEnablement The features enabled for the current run
* @param languagesInput Languages input from the workflow
* @param repository The owner/name of the repository
* @param logger A logger
* @returns A boolean indicating whether or not the toolcache should be bypassed and the latest codeql should be downloaded.
*/
export async function shouldBypassToolcache(
featuresEnablement: FeatureEnablement,
codeqlUrl: string | undefined,
languagesInput: string | undefined,
repository: RepositoryNwo,
logger: Logger
): Promise<boolean> {
// An explicit codeql url is specified, that means the toolcache will not be used.
if (codeqlUrl) {
return true;
}
// Check if the toolcache is disabled for all languages
if (await featuresEnablement.getValue(Feature.BypassToolcacheEnabled)) {
return true;
}
// Check if the toolcache is disabled for kotlin and swift.
if (
!(await featuresEnablement.getValue(
Feature.BypassToolcacheKotlinSwiftEnabled
))
) {
return false;
}
// Now check to see if kotlin or swift is one of the languages being analyzed.
const { rawLanguages, autodetected } = await getRawLanguages(
languagesInput,
repository,
logger
);
let bypass = rawLanguages.some((lang) => KOTLIN_SWIFT_BYPASS.includes(lang));
if (bypass) {
logger.info(
`Bypassing toolcache for kotlin or swift. Languages: ${rawLanguages}`
);
} else if (!autodetected && rawLanguages.includes(Language.java)) {
// special case: java was explicitly specified, but there might be
// some kotlin in the repository, so we need to make a request for that.
const langsInRepo = await getLanguagesInRepo(repository, logger);
if (langsInRepo.includes("kotlin")) {
logger.info(`Bypassing toolcache for kotlin.`);
bypass = true;
}
}
return bypass;
}
export function parseMatrixInput(
matrixInput: string | undefined
): { [key: string]: string } | undefined {