# codeql-mrva Run one CodeQL query across up to 1,000 GitHub repositories at once with Multi-Repository Variant Analysis (MRVA), from VS Code, the gh-mrva CLI, the REST API, or locally against downloaded CodeQL databases. Covers the whole variant-analysis loop - turn a known bug or CVE into a precise query, test it on one database, build a target repository list (top-N, org, code search), set up the controller repository, submit, poll, download SARIF, triage and report responsibly. Use it whenever the user mentions MRVA, variant analysis, "run this CodeQL query across many repos", "find other projects with this bug pattern", a controller repository, gh mrva, CodeQL databases from GitHub, or wants to hunt one vulnerability class across an org or ecosystem, even without the word MRVA. - **Kind:** skill - **Source:** https://github.com/TikoTikTok/codeql-mrva - **Page:** https://forefy.com/skills/ea5a5772-0884-49b1-bc31-c006dcbd84d8 - **API (JSON + files):** https://forefy.com/api/asr/ea5a5772-0884-49b1-bc31-c006dcbd84d8 ## Install Ask your agent: `Install the codeql-mrva skill securely from: https://forefy.com/skills/ea5a5772-0884-49b1-bc31-c006dcbd84d8 globally to ~/.claude/skills` Or run the pinned install script: `curl -fsSL https://forefy.com/api/asr/ea5a5772-0884-49b1-bc31-c006dcbd84d8/install | bash` (`?location=project` for the current project, `&harness=agents` for `.agents/`, `&shell=powershell` with `irm ... | iex`). --- ## SKILL.md --- name: codeql-mrva description: Run one CodeQL query across up to 1,000 GitHub repositories at once with Multi-Repository Variant Analysis (MRVA), from VS Code, the gh-mrva CLI, the REST API, or locally against downloaded CodeQL databases. Covers the whole variant-analysis loop - turn a known bug or CVE into a precise query, test it on one database, build a target repository list (top-N, org, code search), set up the controller repository, submit, poll, download SARIF, triage and report responsibly. Use it whenever the user mentions MRVA, variant analysis, "run this CodeQL query across many repos", "find other projects with this bug pattern", a controller repository, gh mrva, CodeQL databases from GitHub, or wants to hunt one vulnerability class across an org or ecosystem, even without the word MRVA. license: MIT metadata: version: 1.0.0 author: TikoTikTok hermes: tags: [codeql, mrva, variant-analysis, security-research, sast, github] category: security --- # CodeQL multi-repository variant analysis (MRVA) Variant analysis is the step after a bug is found: model it as a CodeQL query, then run that query everywhere the same pattern can live. MRVA is GitHub's hosted way to do the "everywhere" part. You hand it a query pack and a list of up to 1,000 repositories; a dynamic GitHub Actions workflow in a *controller repository* you own fans out one job per repository, runs the query against the CodeQL database GitHub already stores for that repository's default branch, and streams SARIF results back as each job finishes. Nothing is cloned or built on your machine, and GitHub stores databases for thousands of public repositories (every one that runs code scanning with CodeQL, plus many that do not), so a first run usually works the first time. Treat it as a precision instrument, not a search engine. A query that is 90 % noise on one repository is 900 findings of noise on 1,000. The loop below spends most of its effort before the big run, on making the query tight. ## Choose how to run it | Situation | Use | Read | |---|---|---| | Iterating on a query, want to click through results | VS Code CodeQL extension, "Run Variant Analysis" | `references/vscode-workflow.md` | | Scripted or repeated runs, want SARIF on disk | `gh mrva` CLI extension (unofficial, GitHub Security Lab) | `references/gh-mrva-cli.md` | | Own automation, CI, no VS Code | REST API `POST /repos/{controller}/code-scanning/codeql/variant-analyses` | `references/how-mrva-works.md` | | Under ~50 repositories, offline, custom CLI flags, no controller repo | Download databases and run `codeql` locally (`mrva` by Trail of Bits, or by hand) | `references/local-variant-analysis.md` | Hosted MRVA needs: a GitHub token with the `repo` scope (what `gh auth login` and the VS Code sign-in give you), a controller repository with at least one commit, and a query inside a CodeQL pack. The CodeQL CLI is needed by `gh mrva` (it bundles the pack) and for local testing; the VS Code extension manages its own copy. Languages the hosted service accepts: `actions`, `cpp`, `csharp`, `go`, `java` (incl. Kotlin), `javascript` (incl. TypeScript), `python`, `ruby`, `rust`, `swift`. One language per run. ## The loop ### 1. Frame the variant before writing QL Start from the thing that is known to be wrong: a CVE, an advisory, a fix commit, a bug you just found. Write down one line each: - **sink**: the dangerous operation (`pickle.loads`, `exec`, a shell call, a path join that reaches `open`, a deserializer, an `innerHTML` assignment) - **source**: where attacker data enters (HTTP params, headers, file contents, env, argv; a library's public API when the library is the target) - **sanitizer / barrier**: what the fix added (an allowlist, an escape, a type check); its absence is what makes a variant a variant - **what counts as a true positive** in a repository you have never seen, so that triage later is a checklist, not a judgment call The fix diff is the best spec: the lines it removed are the sink pattern, the guard it added is the barrier. ### 2. Write the query as a pack, sink-first Copy `assets/qlpack-template/` (a `qlpack.yml` plus a Python `path-problem` example) and rename the pack. Two rules that save hours: - Write a **sink-only `@kind problem` query first**. It is cheap, tells you how common the API is across the ecosystem, and checks that your model of the sink matches real code. Promote it to `@kind path-problem` with `TaintTracking::Global` only once the sink predicate is right. - Keep the **required metadata**: `@name`, `@id` (unique, e.g. `py/variant/`), `@kind` (`problem` or `path-problem`), `@problem.severity`. A pack whose query does not compile fails the whole run, and `path-problem` is what gives the "Show paths" view in results. `references/query-authoring.md` has the modern dataflow skeleton for each language, the metadata table, pack dependency resolution (`codeql pack install`) and the precision-tuning checklist. Mine the Security Lab's community packs (`githubsecuritylab/codeql--queries`) for sinks and sources before modelling your own. ### 3. Test on one database before 1,000 ```bash python scripts/mrva_dbs.py owner/repo --language python --download dbs/ --extract codeql database analyze dbs/owner__repo__python my-pack/UnsafePickleVariant.ql \ --format=sarif-latest --output=one.sarif --download python scripts/mrva_sarif.py one.sarif ``` Pick two or three repositories: the one where the bug is known to exist, and a popular one that should be clean. Zero results on the known-bad one means the model is wrong; a flood on the clean one means the sink is too wide or a barrier is missing. Only then scale. ### 4. Build the target list ```bash python scripts/mrva_list.py top --language python --limit 500 --name py-top500 --out databases.json python scripts/mrva_list.py org rails --name rails-org --out databases.json python scripts/mrva_list.py search 'pickle.loads language:python' --name py-pickle --out databases.json python scripts/mrva_dbs.py --file databases.json --list py-pickle --language python \ --write-list py-pickle-db --out databases.json ``` The script writes the `gh-mrva` list format by default and the VS Code `databases.json` schema with `--format vscode`, dedupes, warns above 1,000, and `--chunk` splits bigger lists into `name-1`, `name-2`, ... A code-search list is the highest-signal start: it only contains repositories that mention your sink. Then filter it through `mrva_dbs.py` so the run is not full of "no database" skips. `references/target-lists.md` covers the search qualifiers, the built-in Top 10/100/1000 lists, org-wide lists, dependents and the 1,000-per-search cap. ### 5. Set up the controller repository once ```bash bash scripts/mrva_controller.sh OWNER/mrva-controller # public: public targets only, free bash scripts/mrva_controller.sh OWNER/mrva-controller --private # required for private/internal targets ``` It creates the repository with one commit (an empty repository fails with "does not have a branch") and can set the `MRVA_RUNNER_OS` Actions variable for self-hosted runners. A public controller may only analyse public repositories and costs nothing; a private controller may analyse private and internal ones, and Actions minutes beyond the free tier are billed to its owner. Point VS Code at it with `codeQL.variantAnalysis.controllerRepo`, or `gh mrva` with `controller:` in `~/.config/gh-mrva/config.yml` (see `assets/gh-mrva.config.example.yml`). ### 6. Submit VS Code: select the list in "Variant Analysis Repositories", right-click in the `.ql` file, **CodeQL: Run Variant Analysis**. CLI: ```bash gh mrva submit --language python --session pickle-2026-10 \ --list py-pickle-db --query my-pack/UnsafePickleVariant.ql gh mrva status --session pickle-2026-10 gh mrva download --session pickle-2026-10 --output-dir results/pickle-2026-10 ``` Name sessions after the query and date; `gh mrva list` is the only index of them. Lists above the per-run limit are split into several runs under one session automatically. When a repository fails, its workflow log is under the controller repository's Actions tab. ### 7. Triage like a reviewer, not a scanner ```bash python scripts/mrva_sarif.py results/pickle-2026-10 --repo-list hits --out databases.json > triage.md ``` That gives per-repository and per-rule counts, every finding as `repo path:line message`, and a `hits` list of only the repositories with results (what VS Code's **Copy repository list** does) so the refined query can be re-run on exactly those. Then, per finding: 1. Open the file at the commit the database was built from (`database_commit_sha` in the run, or the link VS Code gives), not `HEAD`; the code may have moved. 2. Walk the path for `path-problem` results; the first and last steps usually decide it. 3. Classify: true positive / unreachable source / sanitised elsewhere / test or example code / dead code. Everything but the first is query feedback. 4. Fold the feedback into the query (a barrier, a narrower source, a path filter such as `not f.getRelativePath().matches("%test%")`) and re-run on `hits`. `references/triage-and-disclosure.md` has the SARIF field map, the prioritisation heuristics (stars, last push, whether the sink is reachable from the package's public surface) and a findings-tracker template. ### 8. Report responsibly The analysis is read-only, but the output is a list of probably real vulnerabilities in other people's code. Default to GitHub private vulnerability reporting on each repository (or the contact in `SECURITY.md`), one report per repository, with the query attached so maintainers can re-run it, and a disclosure date (90 days is the Security Lab norm). Do not publish the raw SARIF or a list of affected repositories before fixes ship, and keep anything from private repositories inside the organisation that owns them. For bug-bounty work, check program scope before the run, not after. ## Limits and gotchas - **1,000 repositories per run**, one language per run, and only repositories that already have a CodeQL database; the rest are reported as skipped, not analysed. `mrva_dbs.py` tells you in advance. - The controller repository must have at least one commit so its default branch exists. Public controller for public targets; private controller for anything private or internal. Keep it free of code: its Actions logs are where failures show up, and anyone with write access can read them. - Built-in Top 10/100/1000 lists exist on GitHub.com only (not `*.ghe.com`). Code-search lists add at most 1,000 repositories per search. - The pack is compiled by the action with its own CodeQL version; a query that compiles only on a bleeding-edge local CLI fails remotely. Depend on `codeql/-all` with a range, not an exact old version. - `gh mrva` is an unofficial Security Lab tool; its latest release is v1.0.8 (October 2023) and it shells out to a `codeql` binary on `PATH`. If it breaks, the REST calls in `references/how-mrva-works.md` are what it does. - Stored databases cover the **default branch** only and are refreshed when code scanning runs; some are months old. Verify against `HEAD` before reporting. - Results are Actions artifacts of the controller run and follow that repository's artifact retention; download what you need to keep. ## Files in this skill | Read when | File | |---|---| | Architecture, REST API, billing, database availability, VS Code setting IDs | `references/how-mrva-works.md` | | Click-by-click VS Code workflow, `databases.json` schema, code-search lists, export | `references/vscode-workflow.md` | | `gh mrva` install, config, every flag, output file naming | `references/gh-mrva-cli.md` | | The same analysis locally: downloading databases, Trail of Bits `mrva`, `codeql database analyze` | `references/local-variant-analysis.md` | | Writing and tuning the query, pack layout, metadata, per-language dataflow skeletons | `references/query-authoring.md` | | Building and filtering repository lists | `references/target-lists.md` | | SARIF triage, prioritisation, disclosure | `references/triage-and-disclosure.md` | | Installing this skill into Claude Code, Hermes Agent, Codex | `references/install.md` | Scripts (Python 3.9+, standard library only; `gh` does anything remote): | Script | Does | |---|---| | `scripts/mrva_dbs.py` | Which repositories have a CodeQL database (per language, size, age); filter a list to them; download and extract databases | | `scripts/mrva_list.py` | Build lists from top-N by stars, an owner, a code search, or files; merge, dedupe, cap, chunk, convert between gh-mrva and VS Code formats | | `scripts/mrva_sarif.py` | Summarise SARIF files (from `gh mrva download` or local runs) into a triage table and a hits list | | `scripts/mrva_controller.sh` | Create a controller repository with its first commit; optional self-hosted runner labels | Assets: `assets/qlpack-template/qlpack.yml`, `assets/qlpack-template/UnsafePickleVariant.ql`, `assets/qlpack-template/PickleLoadsSinks.ql`, `assets/qlpack-template/README.md`, `assets/databases.example.json`, `assets/gh-mrva.config.example.yml`. ## assets ``` ``` ## assets/databases.example.json ```json { "py-pickle": [ "psf/requests", "pallets/flask", "django/django" ], "flatbuffers": [ "google/flatbuffers" ] } ``` ## assets/gh-mrva.config.example.yml ```yaml # ~/.config/gh-mrva/config.yml (Windows: C:\Users\\.config\gh-mrva\config.yml) # Every key can be overridden per command: --controller, --list-file, --codeql-path. # NWO of the controller repository (create it with scripts/mrva_controller.sh) controller: OWNER/mrva-controller # JSON file with the repository lists: {"list-name": ["owner/repo", ...]} # (scripts/mrva_list.py writes this format by default) list_file: /abs/path/to/databases.json # Optional. A checkout of https://github.com/github/codeql at the tag that matches # your CLI (git checkout codeql-cli/vX.Y.Z). Only needed when the query pack does # not declare its own dependencies in qlpack.yml. # codeql_path: /abs/path/to/codeql ``` ## assets/qlpack-template ``` ``` ## assets/qlpack-template/PickleLoadsSinks.ql ``` ``` ## assets/qlpack-template/README.md # Query pack template Copy this folder, rename the pack in `qlpack.yml` (`/`, lowercase), swap `codeql/python-all` for the library of your language, then: ```bash codeql pack install . # writes codeql-pack.lock.yml; commit it codeql database analyze PickleLoadsSinks.ql --format=sarif-latest --output=sinks.sarif codeql database analyze UnsafePickleVariant.ql --format=sarif-latest --output=flow.sarif ``` `PickleLoadsSinks.ql` is the sink-only query (step 1), `UnsafePickleVariant.ql` the taint-tracking variant (step 2). Both carry the metadata MRVA needs (`@name`, `@id`, `@kind`, `@problem.severity`). ## assets/qlpack-template/UnsafePickleVariant.ql ``` ``` ## assets/qlpack-template/qlpack.yml ```yaml name: yourname/mrva-python-variants version: 0.0.1 library: false dependencies: codeql/python-all: "*" ``` ## references ``` ``` ## references/gh-mrva-cli.md # gh-mrva: MRVA from the command line `gh-mrva` is a GitHub CLI extension by GitHub Security Lab (https://github.com/GitHubSecurityLab/gh-mrva). It is unofficial and not supported by GitHub; the latest release is v1.0.8 (October 2023) with prebuilt binaries for Linux, macOS, Windows (amd64/arm64) and more. It does the same REST calls as the VS Code extension and writes SARIF to disk, which makes it the right tool for scripted, repeated or large runs. ## Install ```bash gh extension install GitHubSecurityLab/gh-mrva gh mrva --help ``` It shells out to a `codeql` binary on `PATH` for `codeql pack install` and `codeql pack bundle`. The quickest way to get one: ```bash gh extension install github/gh-codeql # downloads the CLI on first use gh codeql install-stub # puts a `codeql` shim on PATH (prints the directory to add) codeql version ``` or download a release of `github/codeql-cli-binaries` and add it to `PATH`. ## Configure `~/.config/gh-mrva/config.yml` (Windows: `C:\Users\\.config\gh-mrva\config.yml`): ```yaml controller: OWNER/mrva-controller # NWO of the controller repository list_file: /abs/path/to/databases.json # repository lists, see below # codeql_path: /abs/path/to/github-codeql-checkout # optional: checkout of github/codeql at the tag # # matching your CLI (codeql-cli/vX.Y.Z); only needed # # when your pack does not declare its dependencies ``` Every option can be overridden per command with `--controller`, `--list-file` and `--codeql-path`. A template is at `assets/gh-mrva.config.example.yml`. ## Repository list file A flat JSON object: list name to array of `owner/repo`: ```json { "py-pickle": ["psf/requests", "pallets/flask"], "rails-org": ["rails/rails", "rails/thor"] } ``` This is not the VS Code `databases.json` schema; `scripts/mrva_list.py` converts between the two (`--format gh-mrva` is the default). ## Commands ### submit ```bash gh mrva submit --language --session --list \ [--list-file ] [--controller ] [--codeql-path ] \ (--query | --query-suite ) \ [--additional-packs ] [--action-branch main] ``` | Flag | Short | Meaning | |---|---|---| | `--session` | `-s` | required; name you will use for `status`/`download`; sessions are kept in a YAML file next to the config | | `--language` | `-l` | required; `actions cpp csharp go java javascript python ruby rust swift` | | `--query` | `-q` | path to one `.ql` file (inside a pack with `qlpack.yml`) | | `--query-suite` | `-x` | path to a `.qls` suite instead of one query | | `--list` | `-i` | name of the list inside the list file | | `--list-file` | `-f` | overrides `list_file` | | `--controller` | `-c` | overrides `controller` | | `--codeql-path` | `-p` | overrides `codeql_path` | | `--additional-packs` | `-a` | extra pack search path passed to `codeql pack` | | `--action-branch` | `-b` | branch of `github/codeql-variant-analysis-action` (default `main`) | The query's pack is installed and bundled, Base64-encoded, and the list is split into chunks sized to the API limit; each chunk is one variant-analysis run, all recorded under the session name. ### status ```bash gh mrva status --session [--json] ``` Per run and per repository: `analysis_status`, result counts, failures. Re-run until every repository is `succeeded`, `failed`, `canceled` or `timed_out`. ### download ```bash gh mrva download --session --output-dir [--download-dbs] [--nwo owner/repo] ``` Writes, per repository with an artifact: ``` __.sarif __.bqrs ____db.zip # only with --download-dbs ``` (`/` in the NWO becomes `_`; GitHub owner names cannot contain `_`, so the first underscore always separates owner from repo.) `--nwo` restricts the download to one repository. Existing files are skipped, so the command is safe to re-run while jobs finish. `scripts/mrva_sarif.py ` summarises the directory. ### list ```bash gh mrva list [--json] ``` Every session with its runs and controller. ## A complete session ```bash gh mrva submit -l python -s pickle-2026-10 -i py-pickle-db -q ./my-pack/UnsafePickleVariant.ql gh mrva status -s pickle-2026-10 gh mrva download -s pickle-2026-10 --output-dir results/pickle-2026-10 python scripts/mrva_sarif.py results/pickle-2026-10 --repo-list pickle-hits --out databases.json > triage.md # tighten the query, then: gh mrva submit -l python -s pickle-2026-10-v2 -i pickle-hits -q ./my-pack/UnsafePickleVariant.ql ``` ## Known rough edges - Unofficial; an API change on GitHub's side can break it without a release. The REST calls it makes are in `references/how-mrva-works.md`. - The README calls the config key `codeql_path` and the note `codeql-dist`; the flag is `--codeql-path`. - It needs `codeql` on `PATH` even though the analysis itself runs on GitHub. - No chunk-size flag; lists above the limit are chunked for you. - Windows: paths in `config.yml` with backslashes must be quoted; forward slashes work. ## references/how-mrva-works.md # How MRVA works Multi-repository variant analysis (MRVA) was announced as a public beta on 9 March 2023 (GitHub blog, Walker Chabbott and James Fletcher: "Multi-repository variant analysis: a powerful new way to perform security research across GitHub"). GitHub's Security Lab had already used it to find the ownCloud Android advisories GHSL-2022-059 / GHSL-2022-060, which affected applications with more than 10 million installs. Code-search-built repository lists were added in June 2023. The feature is part of the CodeQL for VS Code extension and the code scanning REST API; it is free for public repositories. ## Pipeline 1. **Pack**. The client (VS Code extension, `gh mrva`, or your script) takes the query, resolves its library dependencies, and bundles everything into one CodeQL pack tarball: `codeql pack install ` then `codeql pack bundle -o pack.tgz `. 2. **Submit**. The tarball is Base64-encoded and posted, with the language and the target list, to the controller repository: ``` POST /repos/{owner}/{controller}/code-scanning/codeql/variant-analyses { "language": "python", // actions cpp csharp go java javascript python ruby rust swift "query_pack": "", "repositories": ["owner/repo", ...] // exactly one of these three "repository_lists": ["top_1000"], // system list names "repository_owners": ["rails"] // every repo of an org/user (1,000 cap) } ``` The token needs the `repo` scope. The response is `201` with the variant-analysis `id`, `controller_repo`, `actor`, `query_language`, `query_pack_url`, `status`, `actions_workflow_run_id`, `scanned_repositories[]` and `skipped_repositories{}`. 3. **Fan-out**. GitHub creates a dynamic workflow run in the controller repository (you never write it; it checks out `github/codeql-variant-analysis-action`). One parallel job per target repository downloads that repository's stored CodeQL database, runs the query, and uploads `results.sarif` and `results.bqrs` as an artifact. 4. **Collect**. Clients poll: ``` GET /repos/{owner}/{controller}/code-scanning/codeql/variant-analyses/{id} GET /repos/{owner}/{controller}/code-scanning/codeql/variant-analyses/{id}/repos/{repo_owner}/{repo_name} ``` The per-repository response carries `analysis_status` (`pending`, `in_progress`, `succeeded`, `failed`, `canceled`, `timed_out`), `result_count`, `artifact_size_in_bytes`, `artifact_url` (a zip with the SARIF and BQRS), `database_commit_sha` (the commit the database was built from; use it when you open files) and `source_location_prefix`. Reading needs `security_events` (private) or `public_repo` (public). `skipped_repositories` tells you why a target was not analysed: `access_mismatch_repos` (visibility does not match the controller), `not_found_repos`, `no_codeql_db_repos`, `over_limit_repos` (beyond 1,000). ## Controller repository - Any repository you can push to. It can be empty of code, but it **must have at least one commit** so a default branch exists; otherwise the run fails with "does not have a branch". - **Public** controller (GitHub.com): may analyse public repositories only; Actions minutes are free. - **Private** controller: required to analyse any private or internal repository; Actions minutes above the free allowance are billed to the repository owner. - Workflow logs for every run (and the failures) are under its Actions tab. Anyone with read access to the controller sees which repositories you are querying, so do not share a controller with people who should not know. - Self-hosted runners: create the Actions **repository variable** `MRVA_RUNNER_OS` with a JSON list of labels, e.g. `["self-hosted", "linux", "x64"]`. It must be a repository variable, not a secret, an environment variable or anything in a workflow file. No workflow changes are needed. - One controller can be shared by VS Code, `gh mrva` and API scripts. ## Which repositories can be analysed Only repositories that have a stored CodeQL database. GitHub builds and keeps the latest database for the **default branch** of: - every repository that runs code scanning with CodeQL (default or advanced setup), public or private (private needs read access and the right controller visibility); - thousands of additional popular public repositories that do not run code scanning at all. This set changes over time. Check before running: ```bash gh api repos/OWNER/REPO/code-scanning/codeql/databases \ --jq '.[] | "\(.language) \(.size/1048576|floor)MB \(.updated_at) \(.commit_oid)"' ``` A `404` or `403` means no database. Download one (zip) with `-H "Accept: application/zip"`; the response is a `302` to blob storage, which `gh api` and `curl -L` follow. `scripts/mrva_dbs.py` wraps both. ## Limits | Limit | Value | |---|---| | Repositories per run | 1,000 | | Languages per run | 1 | | Repositories added per code search | 1,000 | | Built-in lists | Top 10, Top 100, Top 1000 per language, GitHub.com only | | Result retention | Actions artifacts of the controller run (repository artifact retention applies) | ## VS Code setting IDs | Setting | Default | Meaning | |---|---|---| | `codeQL.variantAnalysis.controllerRepo` | "" | `owner/repo` of the controller | | `codeQL.variantAnalysis.defaultResultsFilter` | `all` | results view filter | | `codeQL.variantAnalysis.defaultResultsSort` | `numberOfResults` | results view sort | | `codeQL.cli.executablePath` | "" | use an existing CodeQL CLI instead of the managed one | | `codeQL.githubDatabase.download` | `ask` | offer to download a repository's database when a workspace opens | The repository lists live in `databases.json` under the workspace storage of the extension (open with the `{ }` button of the "Variant Analysis Repositories" view); the schema is in `references/vscode-workflow.md`. ## Driving the API by hand ```bash codeql pack install my-pack codeql pack bundle -o my-pack.tgz my-pack PACK=$(base64 -w0 my-pack.tgz) # macOS: base64 < my-pack.tgz | tr -d '\n' printf '{"language":"python","query_pack":"%s","repositories":["psf/requests","pallets/flask"]}' "$PACK" \ | gh api -X POST repos/OWNER/CONTROLLER/code-scanning/codeql/variant-analyses --input - > run.json ID=$(jq .id run.json) gh api repos/OWNER/CONTROLLER/code-scanning/codeql/variant-analyses/$ID --jq '.status' gh api repos/OWNER/CONTROLLER/code-scanning/codeql/variant-analyses/$ID/repos/psf/requests \ --jq '{analysis_status, result_count, artifact_url, database_commit_sha}' curl -L -o requests.zip "" # results.sarif + results.bqrs inside ``` This is exactly what `gh mrva submit` / `status` / `download` do, plus chunking of lists above the limit and a local sessions file. ## Sources - https://github.blog/security/vulnerability-research/multi-repository-variant-analysis-a-powerful-new-way-to-perform-security-research-across-github/ - https://docs.github.com/en/code-security/concepts/code-scanning/multi-repository-variant-analysis - https://docs.github.com/en/code-security/how-tos/scan-code-for-vulnerabilities/scan-from-vs-code/running-codeql-queries-at-scale-with-multi-repository-variant-analysis - https://docs.github.com/en/rest/code-scanning/code-scanning (CodeQL databases and variant analyses endpoints) - https://github.com/github/codeql-variant-analysis-action (the action the dynamic workflow runs) - https://github.blog/changelog/2023-06-23-use-github-code-search-to-support-security-research-with-multi-repostiory-variant-analysis-for-codeql-beta/ ## references/install.md # Installing this skill The skill folder is `skills/codeql-mrva/` in https://github.com/TikoTikTok/codeql-mrva. It is a plain Agent Skill (`SKILL.md` + `references/` + `scripts/` + `assets/`), so every agent that reads the agentskills.io format can load it. `install.sh` / `install.ps1` at the repository root copy it into all three agents below in one go: ```bash git clone https://github.com/TikoTikTok/codeql-mrva bash codeql-mrva/install.sh # all detected agents, user scope bash codeql-mrva/install.sh --agents claude,codex --link # symlink instead of copy pwsh codeql-mrva/install.ps1 -Agents hermes # native Windows powershell -ExecutionPolicy Bypass -File codeql-mrva\install.ps1 # when scripts are disabled by policy ``` ## Claude Code Plugin (the repository is its own marketplace): ```bash claude plugin marketplace add TikoTikTok/codeql-mrva claude plugin install codeql-mrva@codeql-mrva ``` Or copy the skill folder: ```bash git clone https://github.com/TikoTikTok/codeql-mrva cp -R codeql-mrva/skills/codeql-mrva ~/.claude/skills/ # personal, every project cp -R codeql-mrva/skills/codeql-mrva .claude/skills/ # this repository only, commit it ``` Or with the skills CLI: ```bash npx skills add TikoTikTok/codeql-mrva -a claude-code -g ``` Invoke with `/codeql-mrva ...` or let it trigger on MRVA / variant analysis requests. New top-level skill directories mid-session need `/reload-skills`. ## Hermes Agent From the GitHub path (Hermes fetches `SKILL.md` and every `references/`, `scripts/` and `assets/` file it mentions, scans them, and asks before installing): ```bash hermes skills inspect TikoTikTok/codeql-mrva/skills/codeql-mrva hermes skills install TikoTikTok/codeql-mrva/skills/codeql-mrva --category security hermes skills list ``` Or copy into the skills directory (the source of truth for Hermes): ```bash cp -R codeql-mrva/skills/codeql-mrva ~/.hermes/skills/security/ # Linux / macOS / WSL ``` ```powershell Copy-Item -Recurse codeql-mrva\skills\codeql-mrva "$env:LOCALAPPDATA\hermes\skills\security\" # native Windows ``` Project-local: `./.hermes/skills/codeql-mrva` or `./.agents/skills/codeql-mrva`, then `hermes skills trust` in that checkout. Invoke with `/codeql-mrva ` in the CLI or any messaging platform; the agent can also load it itself through `skill_view("codeql-mrva")`. `npx skills add TikoTikTok/codeql-mrva -a hermes-agent -g` also works. ## Codex (CLI and IDE extension) Codex reads `$HOME/.agents/skills` (user), `$REPO_ROOT/.agents/skills` and `$CWD/.agents/skills` (project), and `/etc/codex/skills` (admin). ```bash npx skills add TikoTikTok/codeql-mrva -a codex -g # or git clone https://github.com/TikoTikTok/codeql-mrva mkdir -p ~/.agents/skills && cp -R codeql-mrva/skills/codeql-mrva ~/.agents/skills/ ``` Or as a plugin: the repository carries `.codex-plugin/plugin.json` and `.agents/plugins/marketplace.json`, so ```bash codex plugin marketplace add TikoTikTok/codeql-mrva ``` then open the Plugins directory, pick the `tikotiktok` marketplace and install `codeql-mrva`. Invoke with `$codeql-mrva` (or `/skills` to browse); restart Codex if a freshly copied skill does not appear. ## Every agent at once ```bash npx skills add TikoTikTok/codeql-mrva --agent '*' -g -y ``` ## What the skill's commands need on the machine | Tool | Install | Used for | |---|---|---| | GitHub CLI, signed in with `repo` scope | https://cli.github.com, `gh auth login` | every remote call the scripts make | | CodeQL CLI | `gh extension install github/gh-codeql && gh codeql install-stub`, or a `github/codeql-cli-binaries` release on `PATH` | local tests, `gh mrva` pack bundling | | gh-mrva | `gh extension install GitHubSecurityLab/gh-mrva` | CLI runs | | VS Code + `GitHub.vscode-codeql` | VS Code Marketplace | interactive runs | | Python 3.9+ | python.org / winget / brew | the three helper scripts (standard library only) | | `mrva` (Trail of Bits), optional | `python -m pip install mrva` | fully local variant analysis | ## references/local-variant-analysis.md # Variant analysis without the hosted service Use the local route when the target set is small (under ~50 repositories), the network is restricted, you need CLI flags MRVA does not expose (`--threads`, `--ram`, evaluator options, custom extractor settings), you want BQRS for custom post-processing, or the repositories have no stored database and you must build one. It is also the fastest loop for tuning a query before a hosted run. ## 1. Get databases ### From GitHub (no build needed) GitHub stores the default-branch database of every repository that runs CodeQL code scanning plus many popular public ones. ```bash # which languages exist, how big, how old python scripts/mrva_dbs.py psf/requests pallets/flask --language python # download + extract into dbs/____/ python scripts/mrva_dbs.py psf/requests pallets/flask --language python --download dbs/ --extract ``` By hand: ```bash gh api repos/psf/requests/code-scanning/codeql/databases/python \ -H "Accept: application/zip" > requests-python.zip codeql database unbundle requests-python.zip --target dbs/ --name requests-python ``` The zip is the output of `codeql database bundle`; `unbundle` restores it to a usable directory (the one containing `codeql-database.yml`). ### Trail of Bits `mrva` (terminal MRVA) https://github.com/trailofbits/mrva (AGPL-3.0) wraps download + analyse + pretty-print: ```bash python -m pip install mrva # or pipx / uv mkdir dbs mrva download --token $GITHUB_TOKEN --language ruby dbs/ top --limit 100 # top by stars mrva download --token $GITHUB_TOKEN --language ruby dbs/ org rails mrva download --token $GITHUB_TOKEN --language ruby dbs/ repo rails/rails mrva download --token $GITHUB_TOKEN --language ruby dbs/ from-file repos.txt mrva analyze dbs/ /path/to/queries -- --rerun --threads=0 # flags after -- go to codeql mrva pprint dbs/ # SARIF to terminal mrva analyze dbs/ ./q --select 'rails/*' --ignore '*/test*' ``` It needs `codeql` on `PATH` and uses `$GITHUB_TOKEN` when `--token` is absent. Prefer `--threads` over running several `mrva analyze` in parallel. ### Build your own (repository without a database) ```bash git clone --depth 1 https://github.com/owner/repo codeql database create dbs/repo-python --language=python --source-root repo # compiled languages need a working build: codeql database create dbs/repo-cpp --language=cpp --source-root repo --command "make -j" ``` Interpreted languages (Python, JavaScript/TypeScript, Ruby) and Java with build mode `none` extract without building. If you then enable CodeQL code scanning on the repository (yours, or a maintainer's), MRVA can reach it next time. ## 2. Run the query ```bash codeql pack install my-pack # resolve qlpack.yml dependencies once for db in dbs/*/; do name=$(basename "$db") codeql database analyze "$db" my-pack/UnsafePickleVariant.ql \ --format=sarif-latest --output="results/$name.sarif" --threads=0 --rerun done python scripts/mrva_sarif.py results/ > triage.md ``` `--sarif-add-file-contents` embeds snippets for offline review; `--sarif-category` tags the run. For raw tables instead of SARIF: ```bash codeql database run-queries dbs/repo-python my-pack/PickleLoadsSinks.ql codeql bqrs decode --format=csv --output=sinks.csv \ dbs/repo-python/results//PickleLoadsSinks.bqrs ``` Any SARIF produced this way is read by `scripts/mrva_sarif.py` like MRVA output; the repository name is taken from `versionControlProvenance` when present, else from the file name. ## 3. Mixed strategy that works well 1. Download 3 databases, iterate on the query locally until precision is good. 2. Run hosted MRVA on the 1,000-repository list. 3. `--download-dbs` only for the repositories with results, and re-run the final query locally with `--sarif-add-file-contents` for the write-up. ## references/query-authoring.md # Writing the variant query ## Pack layout ``` my-pack/ ├── qlpack.yml ├── codeql-pack.lock.yml # written by `codeql pack install`; commit it ├── PickleLoadsSinks.ql # step 1: sink-only, @kind problem └── UnsafePickleVariant.ql # step 2: taint path, @kind path-problem ``` ```yaml # qlpack.yml name: yourname/mrva-python-variants # /, lowercase version: 0.0.1 library: false dependencies: codeql/python-all: "*" # a range the action's CLI can satisfy; never an old exact pin ``` Library pack names: `codeql/actions-all`, `codeql/cpp-all`, `codeql/csharp-all`, `codeql/go-all`, `codeql/java-all`, `codeql/javascript-all`, `codeql/python-all`, `codeql/ruby-all`, `codeql/rust-all`, `codeql/swift-all`. Run `codeql pack install my-pack` after editing dependencies; MRVA bundles the lock file with the query so the remote compile matches the local one. ## Query metadata ```ql /** * @name Unsafe pickle.loads of request data (variant of CVE-XXXX-YYYY) * @description Untrusted bytes reach pickle.loads without a signature check. * @kind path-problem // problem | path-problem * @problem.severity error // error | warning | recommendation * @security-severity 9.8 // CVSS-like, drives code scanning severity * @precision medium // low | medium | high | very-high * @id py/variant/unsafe-pickle-request // unique; /<...> * @tags security * external/cwe/cwe-502 */ ``` `@name`, `@id`, `@kind` and `@problem.severity` are required for MRVA to display the results; `path-problem` queries must `select sink, source, sink, "message"` and import a `PathGraph`. ## Step 1: sink-only query Cheap, high recall, shows you the API's shape across the ecosystem. ```ql /** * @name pickle.loads call sites * @kind problem * @problem.severity warning * @id py/variant/pickle-loads-sinks */ import python import semmle.python.ApiGraphs from API::CallNode call where call = API::moduleImport("pickle").getMember("loads").getACall() select call, "pickle.loads with argument $@.", call.getArg(0), call.getArg(0).toString() ``` Run it on the Top 100: the count per repository tells you whether the pattern is common enough to be worth a taint query and how many results to expect at full scale. ## Step 2: taint query (modern `ConfigSig` API) ### Python ```ql import python import semmle.python.dataflow.new.DataFlow import semmle.python.dataflow.new.TaintTracking import semmle.python.dataflow.new.RemoteFlowSources import semmle.python.ApiGraphs module PickleConfig implements DataFlow::ConfigSig { predicate isSource(DataFlow::Node source) { source instanceof RemoteFlowSource } predicate isSink(DataFlow::Node sink) { sink = API::moduleImport("pickle").getMember("loads").getACall().getArg(0) } predicate isBarrier(DataFlow::Node node) { // the fix: data that went through hmac.compare_digest / a verify() helper node = API::moduleImport("hmac").getMember("compare_digest").getACall().getArg(_) } } module PickleFlow = TaintTracking::Global; import PickleFlow::PathGraph from PickleFlow::PathNode source, PickleFlow::PathNode sink where PickleFlow::flowPath(source, sink) select sink.getNode(), source, sink, "pickle.loads of data from $@.", source.getNode(), "a remote source" ``` ### Java / Kotlin ```ql import java import semmle.code.java.dataflow.TaintTracking import semmle.code.java.dataflow.FlowSources module Config implements DataFlow::ConfigSig { predicate isSource(DataFlow::Node source) { source instanceof ActiveThreatModelSource } predicate isSink(DataFlow::Node sink) { exists(MethodCall mc | mc.getMethod().hasQualifiedName("java.lang", "Runtime", "exec") and sink.asExpr() = mc.getArgument(0) ) } } module Flow = TaintTracking::Global; import Flow::PathGraph from Flow::PathNode source, Flow::PathNode sink where Flow::flowPath(source, sink) select sink.getNode(), source, sink, "Command built from $@.", source.getNode(), "user input" ``` ### JavaScript / TypeScript ```ql import javascript module Config implements DataFlow::ConfigSig { predicate isSource(DataFlow::Node source) { source instanceof RemoteFlowSource } predicate isSink(DataFlow::Node sink) { sink = DataFlow::globalVarRef("eval").getACall().getArgument(0) } } module Flow = TaintTracking::Global; import Flow::PathGraph from Flow::PathNode source, Flow::PathNode sink where Flow::flowPath(source, sink) select sink.getNode(), source, sink, "eval of $@.", source.getNode(), "remote input" ``` ### Go ```ql import go import semmle.go.security.FlowSources module Config implements DataFlow::ConfigSig { predicate isSource(DataFlow::Node source) { source instanceof RemoteFlowSource } predicate isSink(DataFlow::Node sink) { exists(DataFlow::CallNode c | c.getTarget().hasQualifiedName("os/exec", "Command") and sink = c.getArgument(0) ) } } module Flow = TaintTracking::Global; import Flow::PathGraph from Flow::PathNode source, Flow::PathNode sink where Flow::flowPath(source, sink) select sink.getNode(), source, sink, "Command from $@.", source.getNode(), "remote input" ``` ### Ruby ```ql import codeql.ruby.AST import codeql.ruby.DataFlow import codeql.ruby.TaintTracking import codeql.ruby.dataflow.RemoteFlowSources module Config implements DataFlow::ConfigSig { predicate isSource(DataFlow::Node source) { source instanceof RemoteFlowSource } predicate isSink(DataFlow::Node sink) { exists(DataFlow::CallNode c | c.getMethodName() = "system" and sink = c.getArgument(0)) } } module Flow = TaintTracking::Global; import Flow::PathGraph from Flow::PathNode source, Flow::PathNode sink where Flow::flowPath(source, sink) select sink.getNode(), source, sink, "system() with $@.", source.getNode(), "remote input" ``` ### C / C++ ```ql import cpp import semmle.code.cpp.dataflow.new.TaintTracking import semmle.code.cpp.security.FlowSources module Config implements DataFlow::ConfigSig { predicate isSource(DataFlow::Node source) { source instanceof FlowSource } predicate isSink(DataFlow::Node sink) { exists(FunctionCall fc | fc.getTarget().hasGlobalName("system") and sink.asIndirectArgument() = fc.getArgument(0) ) } } module Flow = TaintTracking::Global; import Flow::PathGraph from Flow::PathNode source, Flow::PathNode sink where Flow::flowPath(source, sink) select sink.getNode(), source, sink, "system() with $@.", source.getNode(), "untrusted input" ``` Library names move between CodeQL releases; `codeql query compile` (or opening the file in VS Code) is the authority. When a class is missing, search the standard library: `grep -rn "class RemoteFlowSource" //ql/lib`. ## Where to get sources and sinks instead of inventing them - The language's standard library: `RemoteFlowSource`, `ActiveThreatModelSource`, `FlowSource`, and the `Concepts` modules (`SystemCommandExecution`, `Decoding`, `FileSystemAccess`, `SqlExecution`, `XmlParsing`) that already model hundreds of APIs. `sink instanceof SystemCommandExecution` is often better than naming one function. - The shipped security queries (`codeql/-queries`, `Security/CWE-*`): copy their `Config` and swap the sink. - GitHub Security Lab community packs: `githubsecuritylab/codeql--queries` and `-libs` (https://github.com/GitHubSecurityLab/CodeQL-Community-Packs). Add them as a dependency and extend. - Data extensions (`.model.yml`) for library APIs that are not modelled. ## Precision tuning checklist Each item is a reason a finding showed up that you did not want. Add the fix, re-run on the `hits` list, repeat. | Noise | Fix | |---|---| | Test, example, benchmark, vendored code | `not sink.getLocation().getFile().getRelativePath().regexpMatch("(^|.*/)(tests?|spec|examples?|vendor|node_modules|third_party)/.*")` | | Source is not really attacker-controlled | narrow `isSource` (drop `LocalFlowSource`, env, argv) or use `ActiveThreatModelSource` with the default threat model | | Data goes through the fix's guard elsewhere | `isBarrier` / `isBarrierIn` for the guard call, or `DataFlow::BarrierGuard` | | Sink argument is a constant | `not sink.asExpr() instanceof StringLiteral` (per-language equivalent) | | Same finding repeated per call path | report one per sink: `@kind problem` on the sink with "has flow from a source" message | | Thousands of results in one repository | that repository is a framework defining the sink; exclude it from the list, not from the query | | Timeouts on big repositories | avoid `exists` over all nodes without a binding; reduce `isSource` set; use `DataFlow::Global` (value flow) when taint steps are not needed | `@precision high` only when, on the test databases, the true-positive rate is above roughly 80 % and the remaining noise is explainable. ## Unit tests (optional, worth it for a query you will re-run) ``` my-pack-tests/ ├── qlpack.yml # dependencies: yourname/mrva-python-variants └── UnsafePickle/ ├── UnsafePickle.qlref # "UnsafePickleVariant.ql" ├── test.py # minimal vulnerable + fixed snippets └── UnsafePickle.expected # created by `codeql test run --learn` ``` ```bash codeql test run my-pack-tests --learn # first time codeql test run my-pack-tests # then: fails when results change ``` ## references/target-lists.md # Building the target repository list The list decides what the run can find. The ceiling is 1,000 repositories per run and only repositories with a stored CodeQL database count, so the work is: start wide, filter to "has a database", order by what matters, cut at 1,000, chunk the rest. ## Sources of candidates | Source | How | Notes | |---|---|---| | Built-in Top 10 / 100 / 1000 | VS Code list, or API `repository_lists: ["top_1000"]` | per language, curated with the code search team, GitHub.com only | | Most-starred in a language | `python scripts/mrva_list.py top --language python --limit 1000` | `gh search repos --sort stars`, cap 1,000 | | Everything an org/user owns | `python scripts/mrva_list.py org rails` or API `repository_owners` | archived and forks excluded by default; `--include-archived`, `--include-forks` | | Code that mentions the sink | `python scripts/mrva_list.py search 'pickle.loads language:python'` or VS Code "Add repositories with GitHub code search" | highest signal; 1,000 results per search, default branches only, files under 384 KB, 10 searches/min | | Dependents of a package | GitHub "Used by" (manual), deps.dev `:dependents` endpoint, libraries.io, or a code search for the import line (`"from yaml import load" language:python`) | the realistic way to find "everyone who uses library X" | | Advisory / SBOM data | `gh api /advisories?ecosystem=pip&affects=`; OSV | for "who still ships the vulnerable version" | | A previous run's hits | `python scripts/mrva_sarif.py results/ --repo-list hits --out databases.json` | iterate the query on exactly the repositories that matter | ## Code search qualifiers that pay off The VS Code integration uses the legacy code search syntax; `gh search code` is the same engine. Useful qualifiers: ``` language:python path:*.py filename:settings.py extension:yml org:rails user:someone repo:owner/name in:file size:<10000 ``` Examples: ``` pickle.loads language:python -path:test -path:tests "yaml.load(" language:python NOT "Loader=" Runtime.getRuntime().exec language:java "child_process" "req.query" language:javascript ``` Code search ranks by relevance and returns at most 1,000 results, so two narrower searches (split by `path:` or `org:`) beat one broad one when you need more than 1,000 candidates. ## Filter to repositories with a database ```bash python scripts/mrva_dbs.py --file databases.json --list py-pickle --language python \ --write-list py-pickle-db --out databases.json ``` Prints a table (language, size, days since the database was built) and writes the filtered list. Roughly: popular repositories almost always have one, small ones only if they run code scanning. Anything without a database needs `codeql database create` locally (see `references/local-variant-analysis.md`) or the maintainer enabling code scanning. ## Ordering and cutting When a candidate set is bigger than 1,000, keep what helps the goal: - **Impact**: stars, dependents, downloads; `gh search repos` already sorts by stars. - **Freshness**: `pushed:>2025-01-01` drops abandoned forks; an old database on a dead project is a finding nobody will fix. - **Relevance**: a code-search list over a stars list; the sink has to be there. - **Diversity**: one repository per org when a monorepo would otherwise dominate the results. Then `--chunk 1000` and submit each chunk as its own session. ## File formats gh-mrva (`scripts/mrva_list.py` default): ```json { "py-pickle": ["psf/requests", "pallets/flask"] } ``` VS Code (`--format vscode`), full schema in `references/vscode-workflow.md`: ```json { "version": 1, "databases": { "variantAnalysis": { "repositoryLists": [ { "name": "py-pickle", "repositories": ["psf/requests"] } ], "owners": [], "repositories": [] } }, "selected": { "kind": "variantAnalysisUserDefinedList", "listName": "py-pickle" } } ``` Both are detected automatically when read. Keep the list files in the research repository next to the query: they are the reproducibility record, and a maintainer asking "how did you find us" deserves an answer. ## references/triage-and-disclosure.md # Triage and disclosure ## SARIF, the parts that matter One file per repository (`__.sarif` from `gh mrva download`, or whatever you named local runs). Inside: ``` runs[0].tool.driver.rules[] id, shortDescription, properties.security-severity, properties.precision runs[0].results[] .ruleId your @id .level error | warning | note (from @problem.severity) .message.text the select message with $@ expanded .locations[0].physicalLocation artifactLocation.uri, region.startLine/startColumn/endLine .codeFlows[] for path-problem: threadFlows[0].locations[] = source ... sink .partialFingerprints stable id for dedupe across runs runs[0].versionControlProvenance[] repositoryUri, revisionId (when present) ``` `scripts/mrva_sarif.py` flattens all of that: ```bash python scripts/mrva_sarif.py results/pickle-2026-10 # markdown: per repo, per rule, findings python scripts/mrva_sarif.py results/ --json > findings.json # one object per finding python scripts/mrva_sarif.py results/ --exclude-paths '(^|/)(tests?|spec|examples?|docs?)/' --min-results 1 python scripts/mrva_sarif.py results/ --repo-list hits --out databases.json # repos with results, as a list ``` ## Order of work 1. **Rule-level sanity**: the per-rule table. One rule with 5,000 results is a sink definition that matched a framework; fix the query before reading findings. 2. **Repository-level prioritisation**: results per repository against stars / last push / whether it is a library (its sinks are reachable from every dependent) or an application (only its own inputs). 3. **Finding-level confirmation**, one by one for the top candidates: - open the file at `database_commit_sha` (per-repository API response or the VS Code link); confirm the code is still there on `HEAD`; - for path results, read the first step (is the source really external?) and the last (is the sink really the dangerous overload?); only then the middle; - decide reachability: is there a route from an unauthenticated or low-privilege caller to the source? That is what separates a CVSS 9 from a note. 4. **Classify** every reviewed finding with one of: `TP`, `TP-low` (real but needs privileges / unusual config), `FP-source` (not attacker-controlled), `FP-sanitised` (guarded elsewhere), `FP-dead` (unreachable or test code), `unsure`. The FP categories are query bugs; fix them (see the checklist in `references/query-authoring.md`) and re-run on the `hits` list until the residue is small enough to read in full. ## Findings tracker Keep one row per (repository, finding) from the first review onward; it is the disclosure log later. | repo | file:line | rule | class | severity | reported | channel | response | fixed | advisory | |---|---|---|---|---|---|---|---|---|---| | org/app | api/upload.py:88 | py/variant/unsafe-pickle-request | TP | high | 2026-10-26 | PVR | ack 10-27 | | | `class` is the triage label above; `channel` is private vulnerability reporting (PVR), a `SECURITY.md` contact, or an email. ## Disclosure The run was read-only static analysis; the findings are not. Treat the SARIF of true positives as sensitive from this point. - **One report per repository**, through GitHub private vulnerability reporting when the repository has it (Security tab, "Report a vulnerability"), else the `SECURITY.md` contact, else a maintainer email, else a neutral issue asking for a private channel. Never the full detail in a public issue. - **What to include**: the vulnerable lines with the commit SHA, the source-to-sink path in words, impact and preconditions, a fix suggestion (what the original fix did), the query itself or a link to it so they can re-run it, and your timeline. A proof of concept only when it is safe to write and helps them reproduce. - **Timeline**: say it in the first message. 90 days to public disclosure is the GitHub Security Lab norm; extend for a good-faith maintainer who is shipping a fix. - **Many repositories, one root cause**: if the sink lives in a shared library, report upstream first and let the library's advisory carry the variants; coordinate dates so the first public fix does not expose the rest. For very wide impact a coordinator (the Security Lab, a CERT, the ecosystem's security team) can carry the load. - **Afterwards**: request a CVE through the repository's GitHub advisory, ask for credit, and consider contributing the query to the community packs so the class stays covered. - **Private or internal repositories**: results stay inside the organisation that owns the code and go through its own vulnerability process; do not move them to a public gist or a personal tracker. - **Bug bounties**: scope first. Static findings in repositories outside a program's scope are reports, not submissions. ## references/vscode-workflow.md # MRVA from VS Code, click by click Prerequisite: the **CodeQL** extension (`GitHub.vscode-codeql`) and a workspace that contains your query inside a CodeQL pack (a folder with `qlpack.yml`). The extension downloads and manages its own CodeQL CLI unless `codeQL.cli.executablePath` points at one. ## 1. Controller repository 1. Open the CodeQL view (left bar) and find **Variant Analysis Repositories**. 2. Click **Set up controller repository**, type `owner/repo`, press Enter. 3. Sign in to GitHub when asked (the "GitHub Authentication" prompt opens in VS Code). The value is stored in `codeQL.variantAnalysis.controllerRepo`. The repository must exist and have one commit (`scripts/mrva_controller.sh` creates it). Public controller for public targets, private for anything else. ## 2. Pick targets The view shows, per language, built-in lists **Top 10 / Top 100 / Top 1000** public repositories (GitHub.com only; absent when the controller is on a `*.ghe.com` host), plus whatever you add: - **+** then *From a GitHub repository*: one `owner/repo`. - **+** then *All repositories of GitHub org or owner*: an org or user (the run caps at 1,000). - **Add list** icon: a named custom list; select it and **+** to add repos. - Right-click a custom list, **Add repositories with GitHub code search**: pick a language, type a legacy-syntax code search (`org:rails`, `path:*.py pickle.loads`), Enter. Up to 1,000 repositories per search; Cancel adds none. Lists are stored in `databases.json` in the extension's workspace storage, editable through the `{ }` button: ```json { "version": 1, "databases": { "variantAnalysis": { "repositoryLists": [ { "name": "py-pickle", "repositories": ["psf/requests", "pallets/flask"] } ], "owners": ["rails"], "repositories": ["google/flatbuffers"] } }, "selected": { "kind": "variantAnalysisUserDefinedList", "listName": "py-pickle" } } ``` `selected.kind` is `variantAnalysisSystemDefinedList` for the Top-N lists (`listName`: `top_10`, `top_100`, `top_1000`), `variantAnalysisUserDefinedList` for yours, `variantAnalysisOwner` and `variantAnalysisRepository` for the other two. `scripts/mrva_list.py --format vscode` writes this file, and the list objects it contains are what **Copy repository list** (below) produces, so you can paste between them. ## 3. Run Select the target (click its row), open the `.ql` file, right-click in the editor, **CodeQL: Run Variant Analysis**. The extension builds the pack, posts it, and opens **Variant Analysis Results** as soon as the workflow is scheduled. **Stop query** in that view cancels the run. Packaging or compilation errors appear bottom-right and in the **Problems** view before anything is submitted; run-time problems (a repository without a database, access denied, job failures with a link to the workflow log in the controller repository) appear inside the results view. ## 4. Read results Each row is a repository: result count, visibility, running/finished, star count. Sort by results, stars or last updated; filter to "with results". Click a repository to see each finding with a link to the file on GitHub at the analysed commit. For `path-problem` queries every finding has **Show paths**. Two buttons you will use on every run: - **Export results**: a secret gist or a Markdown file in the workspace; good for handing a maintainer the evidence. - **Copy repository list**: JSON of only the repositories that had results. Paste it into `databases.json` as a new list and re-run the tightened query on just those. ## 5. Where things go wrong | Symptom | Cause / fix | |---|---| | "Controller repository not found" | Repository does not exist, wrong `owner/repo`, or token lacks `repo` scope; sign out and in again | | "does not have a branch 'main'" | Empty controller; push one commit | | Repository listed as skipped | No CodeQL database (enable code scanning with CodeQL on it, or build and analyse locally), or visibility mismatch with the controller | | Query compiles locally, fails remotely | Library version mismatch; pin `codeql/-all` to a range the action's CLI can satisfy, run `codeql pack install` so the lock file is in the pack | | Top 10/100/1000 missing | Controller on GitHub Enterprise host; build a custom list instead | | Very slow / queued | Shared Actions capacity; set `MRVA_RUNNER_OS` to self-hosted labels on the controller | ## Downloading one database into VS Code For local iteration before the big run: CodeQL view, **Databases**, **Download database from GitHub** (cloud icon), type `owner/repo`, choose the language. With `codeQL.githubDatabase.download: ask` the extension also offers this when you open a workspace whose repository has a database. Then run the query with **CodeQL: Run Query on Selected Database** and iterate on precision. ## scripts ``` ``` ## scripts/mrva_controller.sh ```bash #!/usr/bin/env bash # Create (or finish setting up) an MRVA controller repository. # # mrva_controller.sh OWNER/NAME [--private|--public] [--runner-labels '["self-hosted","linux"]'] [--description TEXT] # # A controller repository hosts the dynamic GitHub Actions workflows that MRVA # dispatches. It may be empty of code but must have at least one commit, so this # script creates it with a README. Public controller: public targets only, free. # Private controller: private/internal targets allowed, Actions minutes billed. set -euo pipefail usage() { sed -n '2,10p' "$0" | sed 's/^# \{0,1\}//'; } NWO="" VIS="--public" LABELS="" DESC="MRVA controller: hosts the dynamic workflows of CodeQL multi-repository variant analysis. No code lives here." while (($#)); do case "$1" in --private) VIS="--private" ;; --public) VIS="--public" ;; --runner-labels) LABELS="${2:-}"; shift ;; --description) DESC="${2:-}"; shift ;; -h|--help) usage; exit 0 ;; -*) echo "unknown option: $1" >&2; usage >&2; exit 2 ;; *) NWO="$1" ;; esac shift done if [[ ! "$NWO" =~ ^[A-Za-z0-9-]+/[A-Za-z0-9._-]+$ ]]; then echo "error: give the controller as OWNER/NAME" >&2 usage >&2 exit 2 fi command -v gh >/dev/null 2>&1 || { echo "error: gh (GitHub CLI) not found: https://cli.github.com" >&2; exit 1; } gh auth status >/dev/null 2>&1 || { echo "error: not signed in; run: gh auth login" >&2; exit 1; } if gh repo view "$NWO" --json name >/dev/null 2>&1; then echo "repository $NWO already exists" else gh repo create "$NWO" "$VIS" --add-readme --description "$DESC" >/dev/null echo "created $NWO (${VIS#--})" fi # MRVA fails on an empty repository ("does not have a branch"): make sure one commit exists. if ! gh api "repos/$NWO/commits?per_page=1" >/dev/null 2>&1; then echo "repository has no commits; adding a README" CONTENT=$(printf '# MRVA controller\n\nThis repository only hosts the dynamic GitHub Actions workflows that CodeQL multi-repository variant analysis dispatches. No code lives here.\n' | base64 | tr -d '\n') gh api -X PUT "repos/$NWO/contents/README.md" -f message="MRVA controller" -f content="$CONTENT" >/dev/null fi if [[ -n "$LABELS" ]]; then gh variable set MRVA_RUNNER_OS --repo "$NWO" --body "$LABELS" echo "set Actions variable MRVA_RUNNER_OS=$LABELS" fi BRANCH=$(gh repo view "$NWO" --json defaultBranchRef --jq '.defaultBranchRef.name') VISIBILITY=$(gh repo view "$NWO" --json visibility --jq '.visibility') cat < public targets only (free); private controller -> private/internal targets, Actions minutes billed. MSG ``` ## scripts/mrva_dbs.py ```python #!/usr/bin/env python3 """Which repositories have a CodeQL database on GitHub, and fetch them. mrva_dbs.py owner/repo [owner/repo ...] [--language L] mrva_dbs.py --file LIST [--list NAME] [--language L] [--write-list NAME --out FILE [--format F]] mrva_dbs.py owner/repo --language L --download DIR [--extract] Options --language L CodeQL language id (python, javascript, java, cpp, csharp, go, ruby, rust, swift, actions) --max-age-days N treat databases older than N days as missing when filtering --write-list NAME write the repositories that have a database for --language as a list (needs --out) --download DIR download the --language database zip of each repository that has one --extract after --download, unpack into DIR/____/ (a usable database) --workers N parallel API calls (default 8) --json machine-readable output MRVA only analyses repositories that have a stored CodeQL database for the default branch; everything else is skipped. This script asks GET /repos/{owner}/{repo}/code-scanning/codeql/databases through `gh api`. """ from __future__ import annotations import argparse import datetime as dt import json import os import shutil import subprocess import sys import zipfile from concurrent.futures import ThreadPoolExecutor from typing import Dict, List, Optional, Tuple sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) import mrva_list as ml # noqa: E402 def run_gh(args: List[str], stdout=None) -> Tuple[int, str, str]: try: proc = subprocess.run(["gh", *args], stdout=stdout or subprocess.PIPE, stderr=subprocess.PIPE) except FileNotFoundError: sys.exit("error: `gh` (GitHub CLI) is not on PATH. Install it from " "https://cli.github.com and run `gh auth login`.") out = "" if stdout else proc.stdout.decode("utf-8", "replace") return proc.returncode, out, proc.stderr.decode("utf-8", "replace") def age_days(iso: str) -> Optional[int]: if not iso: return None try: when = dt.datetime.strptime(iso[:19], "%Y-%m-%dT%H:%M:%S").replace(tzinfo=dt.timezone.utc) except ValueError: return None return (dt.datetime.now(dt.timezone.utc) - when).days def fetch_dbs(repo: str) -> dict: rc, out, err = run_gh(["api", f"repos/{repo}/code-scanning/codeql/databases"]) if rc != 0: low = err.lower() if "rate limit" in low: return {"repo": repo, "databases": [], "error": "API rate limit; retry later"} if "(http 404)" in low or "(http 403)" in low or "not found" in low: return {"repo": repo, "databases": [], "error": None} last = err.strip().splitlines()[-1] if err.strip() else "gh api failed" return {"repo": repo, "databases": [], "error": last} try: rows = json.loads(out) if out.strip() else [] except json.JSONDecodeError: return {"repo": repo, "databases": [], "error": "unexpected response"} dbs = [] for row in rows or []: updated = row.get("updated_at") or row.get("created_at") or "" dbs.append({ "language": ml.norm_lang(row.get("language")) or row.get("language"), "size_mb": round((row.get("size") or 0) / 1048576, 1), "updated_at": updated, "age_days": age_days(updated), "commit_oid": (row.get("commit_oid") or "")[:12], }) return {"repo": repo, "databases": dbs, "error": None} def has_db(record: dict, language: Optional[str], max_age: Optional[int]) -> bool: for db in record["databases"]: if language and db["language"] != language: continue if max_age is not None and db["age_days"] is not None and db["age_days"] > max_age: continue return True return False def _safe_extract(zpath: str, target: str) -> None: base = os.path.realpath(target) with zipfile.ZipFile(zpath) as zf: for member in zf.infolist(): dest = os.path.realpath(os.path.join(base, member.filename)) if dest != base and not dest.startswith(base + os.sep): raise RuntimeError(f"refusing to extract {member.filename!r}: path escapes {target}") zf.extractall(base) def _find_db_root(root: str) -> Optional[str]: for dirpath, _dirs, files in os.walk(root): if "codeql-database.yml" in files: return dirpath return None def download(repo: str, language: str, outdir: str, extract: bool) -> Tuple[str, Optional[str]]: outdir = os.path.normpath(outdir) os.makedirs(outdir, exist_ok=True) name = f"{repo.replace('/', '__')}__{language}" zpath = os.path.join(outdir, name + ".zip") target = os.path.join(outdir, name) if extract and os.path.isdir(target): return target, None if not os.path.exists(zpath): tmp = zpath + ".part" with open(tmp, "wb") as fh: rc, _out, err = run_gh(["api", f"repos/{repo}/code-scanning/codeql/databases/{language}", "-H", "Accept: application/zip"], stdout=fh) if rc != 0: os.remove(tmp) return "", err.strip().splitlines()[-1] if err.strip() else "download failed" os.replace(tmp, zpath) if not extract: return zpath, None tmpdir = os.path.join(outdir, f".tmp-{name}") shutil.rmtree(tmpdir, ignore_errors=True) try: _safe_extract(zpath, tmpdir) root = _find_db_root(tmpdir) if root is None: return "", f"{zpath} contains no codeql-database.yml" shutil.move(root, target) finally: shutil.rmtree(tmpdir, ignore_errors=True) return target, None def render_table(records: List[dict], language: Optional[str]) -> str: width = max([len(r["repo"]) for r in records] + [10]) lines = [] if language: lines.append(f"{'repository'.ljust(width)} {language:<10} {'size':>7} {'age':>6} commit") for r in records: if r["error"]: lines.append(f"{r['repo'].ljust(width)} error: {r['error']}") continue dbs = [d for d in r["databases"] if d["language"] == language] if not dbs: lines.append(f"{r['repo'].ljust(width)} {'-':<10} {'':>7} {'':>6} (no {language} database)") continue d = dbs[0] age = f"{d['age_days']}d" if d["age_days"] is not None else "?" lines.append(f"{r['repo'].ljust(width)} {'yes':<10} {str(d['size_mb']) + 'MB':>7} {age:>6} {d['commit_oid']}") else: lines.append(f"{'repository'.ljust(width)} databases (language:size/age)") for r in records: if r["error"]: lines.append(f"{r['repo'].ljust(width)} error: {r['error']}") elif not r["databases"]: lines.append(f"{r['repo'].ljust(width)} (none)") else: cells = ", ".join(f"{d['language']}:{d['size_mb']}MB/{d['age_days']}d" for d in r["databases"]) lines.append(f"{r['repo'].ljust(width)} {cells}") return "\n".join(lines) def main(argv: Optional[List[str]] = None) -> int: ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) ap.add_argument("repos", nargs="*", help="owner/repo") ap.add_argument("--file", help="list file (gh-mrva or VS Code format) or text file") ap.add_argument("--list", dest="list_name", help="list name inside --file") ap.add_argument("--language") ap.add_argument("--max-age-days", type=int) ap.add_argument("--write-list", metavar="NAME") ap.add_argument("--out") ap.add_argument("--format", choices=["gh-mrva", "vscode"]) ap.add_argument("--download", metavar="DIR") ap.add_argument("--extract", action="store_true") ap.add_argument("--workers", type=int, default=8) ap.add_argument("--json", action="store_true") args = ap.parse_args(argv) repos: List[str] = list(args.repos) if args.file: _name, from_file = ml.build_file(args.file, args.list_name) repos += from_file repos = ml.dedupe(repos) if not repos: ap.error("no repositories given (arguments or --file)") language = ml.norm_lang(args.language) if language and language not in ml.LANG_NAMES: ap.error(f"unknown language {args.language!r}; use one of {', '.join(sorted(ml.LANG_NAMES))}") if (args.write_list or args.download) and not language: ap.error("--write-list and --download need --language") if args.write_list and not args.out: ap.error("--write-list needs --out FILE") with ThreadPoolExecutor(max_workers=max(1, args.workers)) as pool: records = list(pool.map(fetch_dbs, repos)) with_db = [r["repo"] for r in records if has_db(r, language, args.max_age_days)] errors = sum(1 for r in records if r["error"]) if args.json: print(json.dumps({"language": language, "with_database": with_db, "records": records}, indent=2)) else: print(render_table(records, language)) sys.stdout.flush() what = f"a {language} database" if language else "at least one database" if args.max_age_days is not None: what += f" newer than {args.max_age_days} days" print(f"{len(with_db)}/{len(records)} repositories have {what}" + (f" ({errors} errors)" if errors else ""), file=sys.stderr) if args.write_list: fmt = ml.write_lists(args.out, {args.write_list: with_db}, args.format) print(f"wrote {args.write_list}: {len(with_db)} repositories -> {args.out} ({fmt})", file=sys.stderr) if args.download: for repo in with_db: path, err = download(repo, language, args.download, args.extract) if err: print(f"{repo}: {err}", file=sys.stderr) else: print(f"{repo}: {path}", file=sys.stderr) if not args.extract: print("unpack with: codeql database unbundle --target (or re-run with --extract)", file=sys.stderr) return 0 if __name__ == "__main__": sys.exit(main()) ``` ## scripts/mrva_list.py ```python #!/usr/bin/env python3 """Build, merge, filter and convert MRVA repository lists. Standard library only; anything remote goes through the GitHub CLI (`gh`). Subcommands top --language L [--limit N] [--query 'stars:>50 pushed:>2025-01-01'] org OWNER [--language L] [--limit N] [--include-archived] [--include-forks] search 'pickle.loads language:python' [--language L] [--limit N] file PATH [--list NAME] an existing list file (gh-mrva or VS Code format) or a text file with one owner/repo per line repos owner/repo [owner/repo ...] Common options --name NAME list name to write (default: derived from the subcommand) --out FILE write/merge into FILE; without it the list is printed as JSON --format F gh-mrva (default) or vscode; auto-detected when --out exists --exclude FILE drop every repository that appears in any list of FILE --chunk N split into NAME-1, NAME-2, ... of at most N repositories --max N warn above N repositories (default 1000, the MRVA per-run limit) --json also print the written list(s) as JSON to stdout Formats gh-mrva : {"list-name": ["owner/repo", ...], ...} vscode : the CodeQL extension's databases.json {"version": 1, "databases": {"variantAnalysis": {"repositoryLists": [ {"name": "...", "repositories": [...]}], "owners": [], "repositories": []}}, "selected": {...}} """ from __future__ import annotations import argparse import json import os import re import subprocess import sys from typing import Dict, Iterable, List, Optional, Tuple MAX_PER_RUN = 1000 GH_SEARCH_CAP = 1000 # GitHub search returns at most 1,000 results per query # CodeQL / MRVA language id -> GitHub "primary language" display names it covers LANG_NAMES = { "cpp": {"C", "C++", "Objective-C", "Objective-C++", "Cuda"}, "csharp": {"C#"}, "go": {"Go"}, "java": {"Java", "Kotlin"}, "javascript": {"JavaScript", "TypeScript", "Vue", "Svelte"}, "python": {"Python"}, "ruby": {"Ruby"}, "rust": {"Rust"}, "swift": {"Swift"}, "actions": set(), # GitHub Actions workflows have no primary-language signal } ALIASES = { "py": "python", "js": "javascript", "ts": "javascript", "typescript": "javascript", "javascript-typescript": "javascript", "c": "cpp", "c++": "cpp", "cxx": "cpp", "c-cpp": "cpp", "c#": "csharp", "cs": "csharp", "golang": "go", "kotlin": "java", "java-kotlin": "java", "rb": "ruby", "rs": "rust", "gha": "actions", "github-actions": "actions", } REPO_RE = re.compile(r"^[A-Za-z0-9](?:[A-Za-z0-9-]{0,38})/[A-Za-z0-9._-]{1,100}$") def norm_lang(lang: Optional[str]) -> Optional[str]: if not lang: return None low = lang.strip().lower() return ALIASES.get(low, low) def norm_repo(value: str) -> Optional[str]: """'https://github.com/Owner/Repo.git/tree/x' -> 'Owner/Repo'; None when not a repository.""" s = (value or "").strip() if not s or s.startswith("#"): return None s = re.sub(r"^(?:https?://)?(?:www\.)?github\.com/", "", s) s = re.sub(r"^git@github\.com:", "", s) parts = [p for p in s.strip("/").split("/") if p] if len(parts) < 2: return None repo = "/".join(parts[:2]) if repo.endswith(".git"): repo = repo[:-4] return repo if REPO_RE.match(repo) else None def dedupe(repos: Iterable[str]) -> List[str]: seen = set() out: List[str] = [] for r in repos: n = norm_repo(r) if n is None: continue key = n.lower() if key in seen: continue seen.add(key) out.append(n) return out # --- gh --------------------------------------------------------------------- def gh(args: List[str]) -> str: try: proc = subprocess.run(["gh", *args], capture_output=True, text=True, encoding="utf-8", errors="replace") except FileNotFoundError: sys.exit("error: `gh` (GitHub CLI) is not on PATH. Install it from " "https://cli.github.com and run `gh auth login`.") if proc.returncode != 0: msg = (proc.stderr or proc.stdout).strip() sys.exit(f"error: gh {' '.join(args[:2])} failed:\n{msg}") return proc.stdout def gh_json(args: List[str]): out = gh(args).strip() return json.loads(out) if out else [] # --- list files ------------------------------------------------------------- def detect_format(doc) -> str: if isinstance(doc, dict): if isinstance(doc.get("databases"), dict): return "vscode" if isinstance(doc.get("name"), str) and isinstance(doc.get("repositories"), list): return "vscode-list" # what "Copy repository list" puts on the clipboard if all(isinstance(v, list) for v in doc.values()): return "gh-mrva" raise ValueError("not a gh-mrva or VS Code repository list") def lists_from_doc(doc) -> Dict[str, List[str]]: fmt = detect_format(doc) if fmt == "gh-mrva": return {k: dedupe(v) for k, v in doc.items()} if fmt == "vscode-list": return {doc["name"]: dedupe(doc["repositories"])} va = doc["databases"].get("variantAnalysis", {}) or {} return { entry["name"]: dedupe(entry.get("repositories", [])) for entry in va.get("repositoryLists", []) or [] if isinstance(entry, dict) and isinstance(entry.get("name"), str) } def read_lists(path: str) -> Dict[str, List[str]]: with open(path, encoding="utf-8-sig") as fh: text = fh.read() if text.lstrip().startswith("{"): return lists_from_doc(json.loads(text)) name = os.path.splitext(os.path.basename(path))[0] return {name: dedupe(text.splitlines())} def _empty_vscode() -> dict: return {"version": 1, "databases": {"variantAnalysis": {"repositoryLists": [], "owners": [], "repositories": []}}, "selected": None} def write_lists(path: str, lists: Dict[str, List[str]], fmt: Optional[str] = None) -> str: """Write or merge `lists` into `path`. Returns the format used.""" doc = None if os.path.exists(path): try: with open(path, encoding="utf-8-sig") as fh: doc = json.load(fh) existing = detect_format(doc) except Exception as exc: # noqa: BLE001 sys.exit(f"error: {path} exists but is not a list file I understand ({exc}); choose another --out") if existing == "vscode-list": # a pasted "Copy repository list" object: promote it to a full file single = doc doc = _empty_vscode() doc["databases"]["variantAnalysis"]["repositoryLists"].append( {"name": single["name"], "repositories": dedupe(single["repositories"])}) existing = "vscode" if fmt and fmt != existing: print(f"note: {path} is in {existing} format; keeping that format", file=sys.stderr) fmt = existing fmt = fmt or "gh-mrva" if fmt == "gh-mrva": if not isinstance(doc, dict): doc = {} for name, repos in lists.items(): doc[name] = repos else: if not isinstance(doc, dict): doc = _empty_vscode() va = doc.setdefault("databases", {}).setdefault("variantAnalysis", {}) rl = va.setdefault("repositoryLists", []) va.setdefault("owners", []) va.setdefault("repositories", []) for name, repos in lists.items(): for entry in rl: if isinstance(entry, dict) and entry.get("name") == name: entry["repositories"] = repos break else: rl.append({"name": name, "repositories": repos}) doc.setdefault("version", 1) doc["selected"] = {"kind": "variantAnalysisUserDefinedList", "listName": list(lists)[-1]} with open(path, "w", encoding="utf-8") as fh: json.dump(doc, fh, indent=2) fh.write("\n") return fmt def vscode_doc(lists: Dict[str, List[str]]) -> dict: doc = _empty_vscode() for name, repos in lists.items(): doc["databases"]["variantAnalysis"]["repositoryLists"].append({"name": name, "repositories": repos}) if lists: doc["selected"] = {"kind": "variantAnalysisUserDefinedList", "listName": list(lists)[-1]} return doc def chunk(name: str, repos: List[str], size: int) -> Dict[str, List[str]]: if size <= 0 or len(repos) <= size: return {name: repos} return {f"{name}-{i // size + 1}": repos[i:i + size] for i in range(0, len(repos), size)} # --- builders --------------------------------------------------------------- def build_top(language: str, limit: int, query: Optional[str]) -> List[str]: lang = norm_lang(language) if lang not in LANG_NAMES: sys.exit(f"error: unknown language {language!r}; use one of {', '.join(sorted(LANG_NAMES))}") limit = min(limit, GH_SEARCH_CAP) args = ["search", "repos"] if query: args.append(query) if lang == "actions": args += ["path:.github/workflows"] if not query else [] else: args += ["--language", lang] args += ["--sort", "stars", "--order", "desc", "--limit", str(limit), "--json", "fullName"] return [row["fullName"] for row in gh_json(args)] def build_org(owner: str, language: Optional[str], limit: int, include_archived: bool, include_forks: bool) -> List[str]: lang = norm_lang(language) rows = gh_json(["repo", "list", owner, "--limit", str(limit), "--json", "nameWithOwner,isArchived,isFork,primaryLanguage"]) out = [] wanted = LANG_NAMES.get(lang or "", set()) for row in rows: if row.get("isArchived") and not include_archived: continue if row.get("isFork") and not include_forks: continue if lang and lang != "actions": primary = (row.get("primaryLanguage") or {}).get("name") if primary not in wanted: continue out.append(row["nameWithOwner"]) return out def build_search(query: str, language: Optional[str], limit: int) -> List[str]: limit = min(limit, GH_SEARCH_CAP) args = ["search", "code", query] lang = norm_lang(language) if lang and lang != "actions": args += ["--language", lang] args += ["--limit", str(limit), "--json", "repository"] rows = gh_json(args) return [row["repository"]["nameWithOwner"] for row in rows if row.get("repository")] def build_file(path: str, list_name: Optional[str]) -> Tuple[str, List[str]]: lists = read_lists(path) if not lists: sys.exit(f"error: no lists found in {path}") if list_name: if list_name not in lists: sys.exit(f"error: list {list_name!r} not in {path}; available: {', '.join(lists)}") return list_name, lists[list_name] if len(lists) == 1: name = next(iter(lists)) return name, lists[name] sys.exit(f"error: {path} holds several lists ({', '.join(lists)}); pick one with --list") # --- cli -------------------------------------------------------------------- def main(argv: Optional[List[str]] = None) -> int: common = argparse.ArgumentParser(add_help=False) common.add_argument("--name", help="list name to write") common.add_argument("--out", help="write/merge into this file") common.add_argument("--format", choices=["gh-mrva", "vscode"], help="output format (default gh-mrva)") common.add_argument("--exclude", help="list file whose repositories are dropped") common.add_argument("--chunk", type=int, default=0, help="split into chunks of N") common.add_argument("--max", type=int, default=MAX_PER_RUN, help="warn above N repositories") common.add_argument("--json", action="store_true", help="print the resulting list(s) as JSON") parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) sub = parser.add_subparsers(dest="cmd", required=True) p = sub.add_parser("top", parents=[common], help="most-starred repositories in a language") p.add_argument("--language", required=True) p.add_argument("--limit", type=int, default=GH_SEARCH_CAP) p.add_argument("--query", help="extra search qualifiers, e.g. 'stars:>50 pushed:>2025-01-01'") p = sub.add_parser("org", parents=[common], help="every repository of an owner") p.add_argument("owner") p.add_argument("--language") p.add_argument("--limit", type=int, default=MAX_PER_RUN) p.add_argument("--include-archived", action="store_true") p.add_argument("--include-forks", action="store_true") p = sub.add_parser("search", parents=[common], help="repositories matched by a code search") p.add_argument("query") p.add_argument("--language") p.add_argument("--limit", type=int, default=GH_SEARCH_CAP) p = sub.add_parser("file", parents=[common], help="repositories from a list or text file") p.add_argument("path") p.add_argument("--list", dest="list_name") p = sub.add_parser("repos", parents=[common], help="literal owner/repo arguments") p.add_argument("repos", nargs="+") args = parser.parse_args(argv) if args.cmd == "top": lang = norm_lang(args.language) name, repos = args.name or f"{lang}-top{min(args.limit, GH_SEARCH_CAP)}", build_top(args.language, args.limit, args.query) elif args.cmd == "org": name = args.name or f"{args.owner}-org" + (f"-{norm_lang(args.language)}" if args.language else "") repos = build_org(args.owner, args.language, args.limit, args.include_archived, args.include_forks) elif args.cmd == "search": name, repos = args.name or "search", build_search(args.query, args.language, args.limit) elif args.cmd == "file": name, repos = build_file(args.path, args.list_name) name = args.name or name else: name, repos = args.name or "repos", args.repos repos = dedupe(repos) if args.exclude: excluded = {r.lower() for lst in read_lists(args.exclude).values() for r in lst} before = len(repos) repos = [r for r in repos if r.lower() not in excluded] print(f"excluded {before - len(repos)} repositories listed in {args.exclude}", file=sys.stderr) lists = chunk(name, repos, args.chunk) for lname, lrepos in lists.items(): if len(lrepos) > args.max: print(f"warning: {lname} has {len(lrepos)} repositories, above the MRVA per-run limit of " f"{args.max}; use --chunk {args.max}", file=sys.stderr) if args.out: fmt = write_lists(args.out, lists, args.format) for lname, lrepos in lists.items(): print(f"wrote {lname}: {len(lrepos)} repositories -> {args.out} ({fmt})", file=sys.stderr) if args.json: print(json.dumps(lists, indent=2)) elif (args.format or "gh-mrva") == "vscode": print(json.dumps(vscode_doc(lists), indent=2)) else: print(json.dumps(lists, indent=2)) return 0 if __name__ == "__main__": sys.exit(main()) ``` ## scripts/mrva_sarif.py ```python #!/usr/bin/env python3 """Summarise SARIF results from `gh mrva download`, VS Code exports or local `codeql database analyze` runs. mrva_sarif.py PATH [PATH ...] files, or directories searched recursively for *.sarif --min-results N only repositories with at least N findings (default 1; 0 lists empty ones) --max-per-repo N findings listed per repository in the markdown (default 25) --exclude-paths REGEX drop findings whose file path matches (e.g. '(^|/)(tests?|spec|examples?)/') --grep REGEX keep findings whose message or path matches --rule ID keep one rule id --json print findings as JSON instead of markdown --repo-list NAME write the repositories with findings as a list (--out FILE, --format F), the same thing VS Code's "Copy repository list" produces The repository name comes from runs[].versionControlProvenance[].repositoryUri when present, else from the file name: gh-mrva writes __.sarif and mrva_dbs.py names databases ____. """ from __future__ import annotations import argparse import json import os import re import sys from collections import Counter, OrderedDict, defaultdict from typing import Dict, Iterable, List, Optional sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) import mrva_list as ml # noqa: E402 def sarif_files(paths: Iterable[str]) -> List[str]: out: List[str] = [] for p in paths: if os.path.isdir(p): for dirpath, _dirs, files in os.walk(p): out += sorted(os.path.join(dirpath, f) for f in files if f.lower().endswith(".sarif")) elif os.path.isfile(p): out.append(p) else: print(f"warning: {p} does not exist", file=sys.stderr) return out def repo_from_run(run: dict, path: str) -> str: for vcp in run.get("versionControlProvenance") or []: uri = vcp.get("repositoryUri") or "" m = re.search(r"github\.com/([^/]+/[^/]+?)(?:\.git)?/?$", uri) if m: return m.group(1) stem = os.path.splitext(os.path.basename(path))[0] if "__" in stem: # owner__repo__language parts = stem.split("__") if len(parts) >= 2: return f"{parts[0]}/{parts[1]}" stem = re.sub(r"_\d+$", "", stem) # owner_repo_ if "_" in stem: # GitHub owners cannot contain '_' owner, repo = stem.split("_", 1) return f"{owner}/{repo}" return stem def _loc(location: dict) -> (str, Optional[int]): phys = (location or {}).get("physicalLocation") or {} uri = (phys.get("artifactLocation") or {}).get("uri") or "" line = (phys.get("region") or {}).get("startLine") return uri, line def load_findings(paths: Iterable[str]) -> (List[dict], Dict[str, int]): """Returns (findings, repositories seen -> result count before filtering).""" findings: List[dict] = [] seen: Dict[str, int] = OrderedDict() for path in sarif_files(paths): try: with open(path, encoding="utf-8-sig") as fh: doc = json.load(fh) except (OSError, json.JSONDecodeError) as exc: print(f"warning: skipping {path}: {exc}", file=sys.stderr) continue for run in doc.get("runs") or []: repo = repo_from_run(run, path) results = run.get("results") or [] seen[repo] = seen.get(repo, 0) + len(results) rules = {r.get("id"): r for r in ((run.get("tool") or {}).get("driver") or {}).get("rules") or []} for res in results: uri, line = _loc((res.get("locations") or [{}])[0]) flows = res.get("codeFlows") or [] path_len = 0 source = "" if flows: locs = ((flows[0].get("threadFlows") or [{}])[0].get("locations") or []) path_len = len(locs) if locs: s_uri, s_line = _loc(locs[0].get("location") or {}) source = f"{s_uri}:{s_line}" if s_line else s_uri rule_id = res.get("ruleId") or (res.get("rule") or {}).get("id") or "?" rule = rules.get(rule_id) or {} findings.append({ "repo": repo, "file": uri, "line": line, "rule": rule_id, "level": res.get("level") or (rule.get("defaultConfiguration") or {}).get("level") or "", "security_severity": (rule.get("properties") or {}).get("security-severity"), "message": re.sub(r"\s+", " ", ((res.get("message") or {}).get("text") or "")).strip(), "path_len": path_len, "source": source, "fingerprint": (res.get("partialFingerprints") or {}).get("primaryLocationLineHash"), "sarif": path, }) return findings, seen def apply_filters(findings: List[dict], exclude_paths: Optional[str], grep: Optional[str], rule: Optional[str]) -> List[dict]: ex = re.compile(exclude_paths) if exclude_paths else None gr = re.compile(grep, re.IGNORECASE) if grep else None out = [] for f in findings: if rule and f["rule"] != rule: continue if ex and ex.search(f["file"] or ""): continue if gr and not (gr.search(f["message"]) or gr.search(f["file"] or "")): continue out.append(f) return out def _cell(text: str, limit: int = 160) -> str: text = (text or "").replace("|", "\\|") return text if len(text) <= limit else text[: limit - 1] + "…" def render_markdown(findings: List[dict], seen: Dict[str, int], min_results: int, max_per_repo: int) -> str: per_repo: Dict[str, List[dict]] = defaultdict(list) for f in findings: per_repo[f["repo"]].append(f) repos = [r for r in seen if len(per_repo.get(r, [])) >= min_results] repos.sort(key=lambda r: (-len(per_repo.get(r, [])), r.lower())) rule_count = Counter(f["rule"] for f in findings) rule_repos: Dict[str, set] = defaultdict(set) for f in findings: rule_repos[f["rule"]].add(f["repo"]) lines = ["# MRVA triage", ""] lines.append(f"- SARIF files: {len({f['sarif'] for f in findings}) if findings else 0} with findings, " f"repositories analysed: {len(seen)}, with findings: {sum(1 for r in seen if per_repo.get(r))}, " f"findings: {len(findings)}") lines += ["", "## Repositories", "", "| repository | findings | rules |", "|---|---:|---|"] for r in repos: rs = sorted({f["rule"] for f in per_repo.get(r, [])}) lines.append(f"| {r} | {len(per_repo.get(r, []))} | {_cell(', '.join(rs), 120)} |") lines += ["", "## Rules", "", "| rule | findings | repositories |", "|---|---:|---:|"] for rule, n in rule_count.most_common(): lines.append(f"| {rule} | {n} | {len(rule_repos[rule])} |") lines += ["", "## Findings", ""] for r in repos: fs = per_repo.get(r, []) if not fs: continue lines += [f"### {r} ({len(fs)})", "", "| location | rule | level | path | message |", "|---|---|---|---:|---|"] for f in sorted(fs, key=lambda x: (x["file"] or "", x["line"] or 0))[:max_per_repo]: loc = f"{f['file']}:{f['line']}" if f["line"] else (f["file"] or "?") path = f"{f['path_len']} ({f['source']})" if f["path_len"] else "" lines.append(f"| {_cell(loc, 100)} | {f['rule']} | {f['level']} | {_cell(path, 80)} | {_cell(f['message'])} |") if len(fs) > max_per_repo: lines.append(f"| … {len(fs) - max_per_repo} more | | | | |") lines.append("") return "\n".join(lines).rstrip() + "\n" def main(argv: Optional[List[str]] = None) -> int: ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) ap.add_argument("paths", nargs="+") ap.add_argument("--min-results", type=int, default=1) ap.add_argument("--max-per-repo", type=int, default=25) ap.add_argument("--exclude-paths", metavar="REGEX") ap.add_argument("--grep", metavar="REGEX") ap.add_argument("--rule") ap.add_argument("--json", action="store_true") ap.add_argument("--repo-list", metavar="NAME") ap.add_argument("--out") ap.add_argument("--format", choices=["gh-mrva", "vscode"]) args = ap.parse_args(argv) findings, seen = load_findings(args.paths) findings = apply_filters(findings, args.exclude_paths, args.grep, args.rule) if args.json: print(json.dumps(findings, indent=2)) else: sys.stdout.write(render_markdown(findings, seen, args.min_results, args.max_per_repo)) if args.repo_list: hits = ml.dedupe(f["repo"] for f in findings) if args.out: fmt = ml.write_lists(args.out, {args.repo_list: hits}, args.format) print(f"wrote {args.repo_list}: {len(hits)} repositories -> {args.out} ({fmt})", file=sys.stderr) else: print(json.dumps({args.repo_list: hits}, indent=2), file=sys.stderr) return 0 if __name__ == "__main__": sys.exit(main()) ```