Unpin CodeQL pack dependencies by removing committed lock files#45
Unpin CodeQL pack dependencies by removing committed lock files#45data-douser merged 8 commits intomainfrom
Conversation
Agent-Logs-Url: https://github.com/advanced-security/codeql-development-template/sessions/a6282c16-1661-4d3b-b8ae-3a50f516d757 Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
data-douser
left a comment
There was a problem hiding this comment.
Review: Additional changes needed before merge
This PR correctly removes committed codeql-pack.lock.yml files and .gitignores future ones — addressing the core issue of lock files pinning pack versions and causing unnecessary churn. However, two follow-up changes are needed to complete this work (per #43):
1. Add scripts/install-codeql-packs.sh
Without lock files, users need a clear way to install all workspace packs after cloning. A script similar to codeql-development-mcp-server's install-packs.sh should be added that:
- Uses
codeql pack ls --format=jsonto dynamically discover packs from the workspace - Iterates and runs
codeql pack installfor each pack directory - Includes retry logic for network resilience
- Supports
--language <lang>filtering
2. Update qlpack.yml workspace references from "*" to ${workspace}
All 16 test packs use "*" for their intra-workspace dependency on the corresponding src pack. Per the CodeQL docs, ${workspace} is the recommended placeholder — it makes the workspace-local nature explicit and resolves correctly during both development and publishing.
See file-level comments for details.
- Add scripts/install-codeql-packs.sh using codeql pack ls
for dynamic pack discovery with retry and --language filter
- Use ${workspace} for intra-workspace deps in test packs
- Bump all 32 workspace packs from 0.0.1 to 0.0.2
- Add pack install step to copilot-setup-steps workflow
- Add README Step 2 for post-clone pack installation
- Revert .gitignore: keep lock files in template-derived repos
Resolves review feedback from #45. See also #43.
There was a problem hiding this comment.
Pull request overview
This PR updates the template’s CodeQL pack dependency strategy by removing committed codeql-pack.lock.yml files (to avoid pinning transitive versions) and introducing automation to (re)generate/install pack dependencies locally/on-CI.
Changes:
- Remove committed
codeql-pack.lock.ymlfiles across language workspaces to unpin transitive pack versions. - Add a helper script to discover workspace packs and run
codeql pack installfor each. - Update per-language
qlpack.ymlversions and adjust test-pack dependencies to reference workspace packs.
Show a summary per file
| File | Description |
|---|---|
| scripts/install-codeql-packs.sh | New script to discover workspace packs and install pack dependencies with retry logic. |
| resources/cli/codeql/codeql_execute_query-server2.prompt.md | Fixes prompt formatting (removes stray fenced block markers; adds spacing). |
| README.md | Adds a setup step instructing users to install CodeQL pack dependencies via the new script. |
| .github/workflows/copilot-setup-steps.yml | Runs pack installation as part of the Copilot setup workflow. |
| languages/actions/tools/test/qlpack.yml | Bumps pack version; switches local dependency reference style. |
| languages/actions/tools/test/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/actions/tools/src/qlpack.yml | Bumps pack version to reflect dependency strategy change. |
| languages/actions/tools/src/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/actions/custom/test/qlpack.yml | Bumps pack version; switches local dependency reference style. |
| languages/actions/custom/test/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/actions/custom/src/qlpack.yml | Bumps pack version to reflect dependency strategy change. |
| languages/actions/custom/src/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/cpp/tools/test/qlpack.yml | Bumps pack version; switches local dependency reference style. |
| languages/cpp/tools/test/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/cpp/tools/src/qlpack.yml | Bumps pack version to reflect dependency strategy change. |
| languages/cpp/tools/src/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/cpp/custom/test/qlpack.yml | Bumps pack version; switches local dependency reference style. |
| languages/cpp/custom/test/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/cpp/custom/src/qlpack.yml | Bumps pack version to reflect dependency strategy change. |
| languages/cpp/custom/src/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/csharp/tools/test/qlpack.yml | Bumps pack version; switches local dependency reference style. |
| languages/csharp/tools/test/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/csharp/tools/src/qlpack.yml | Bumps pack version to reflect dependency strategy change. |
| languages/csharp/tools/src/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/csharp/custom/test/qlpack.yml | Bumps pack version; switches local dependency reference style. |
| languages/csharp/custom/test/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/csharp/custom/src/qlpack.yml | Bumps pack version to reflect dependency strategy change. |
| languages/csharp/custom/src/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/go/tools/test/qlpack.yml | Bumps pack version; switches local dependency reference style. |
| languages/go/tools/test/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/go/tools/src/qlpack.yml | Bumps pack version to reflect dependency strategy change. |
| languages/go/tools/src/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/go/custom/test/qlpack.yml | Bumps pack version; switches local dependency reference style. |
| languages/go/custom/test/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/go/custom/src/qlpack.yml | Bumps pack version to reflect dependency strategy change. |
| languages/go/custom/src/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/java/tools/test/qlpack.yml | Bumps pack version; switches local dependency reference style. |
| languages/java/tools/test/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/java/tools/src/qlpack.yml | Bumps pack version to reflect dependency strategy change. |
| languages/java/tools/src/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/java/custom/test/qlpack.yml | Bumps pack version; switches local dependency reference style. |
| languages/java/custom/test/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/java/custom/src/qlpack.yml | Bumps pack version to reflect dependency strategy change. |
| languages/java/custom/src/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/javascript/tools/test/qlpack.yml | Bumps pack version; switches local dependency reference style. |
| languages/javascript/tools/test/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/javascript/tools/src/qlpack.yml | Bumps pack version to reflect dependency strategy change. |
| languages/javascript/tools/src/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/javascript/custom/test/qlpack.yml | Bumps pack version; switches local dependency reference style. |
| languages/javascript/custom/test/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/javascript/custom/src/qlpack.yml | Bumps pack version to reflect dependency strategy change. |
| languages/javascript/custom/src/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/python/tools/test/qlpack.yml | Bumps pack version; switches local dependency reference style. |
| languages/python/tools/test/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/python/tools/src/qlpack.yml | Bumps pack version to reflect dependency strategy change. |
| languages/python/tools/src/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/python/custom/test/qlpack.yml | Bumps pack version; switches local dependency reference style. |
| languages/python/custom/test/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/python/custom/src/qlpack.yml | Bumps pack version to reflect dependency strategy change. |
| languages/python/custom/src/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/ruby/tools/test/qlpack.yml | Bumps pack version; switches local dependency reference style. |
| languages/ruby/tools/test/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/ruby/tools/src/qlpack.yml | Bumps pack version to reflect dependency strategy change. |
| languages/ruby/tools/src/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/ruby/custom/test/qlpack.yml | Bumps pack version; switches local dependency reference style. |
| languages/ruby/custom/test/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
| languages/ruby/custom/src/qlpack.yml | Bumps pack version to reflect dependency strategy change. |
| languages/ruby/custom/src/codeql-pack.lock.yml | Removes committed lockfile to avoid pinning dependencies. |
Copilot's findings
- Files reviewed: 68/68 changed files
- Comments generated: 3
📝 Query Update
actions,cpp,csharp,go,java,javascript,python,ruby)🎯 Changes
What Changed
codeql-pack.lock.ymlfiles underlanguages/<lang>/{custom,tools}/{src,test}/.codeql-pack.lock.ymlto.gitignoreso locally generated lock files are not re-committed.qlpack.ymlfiles — they already declarecodeql/*-all(and*-queries) as"*".Why This Change
Committed lock files pinned packs to old versions (e.g.
codeql/java-all 7.7.0), blocking template consumers from using features tied to newer libraries — for example, thebarrierModelextensible predicate, which requirescodeql/java-allshipped with CodeQL 2.25.2+.Impact
Consumers of this template generate their own
codeql-pack.lock.ymloncodeql pack install, resolving against whichever CodeQL CLI they have installed. The template no longer dictates a CodeQL version floor/ceiling.🔍 Before vs. After
Previous Behavior
codeql-pack.lock.ymlchecked into the repo pinned every transitive dependency, e.g.:New Behavior
Only
qlpack.ymlis tracked, with floating versions:Lock files are produced locally per consumer and ignored by git.
🧪 Testing
📊 Impact Analysis
barrierModel)📋 Checklist
🔗 References