Skip to content

feat: add speckit.threatmodel command for OWASP Top 10 for LLM applications#2287

Open
NaviaSamal wants to merge 5 commits intogithub:mainfrom
NaviaSamal:feat/add-threat-model-skill
Open

feat: add speckit.threatmodel command for OWASP Top 10 for LLM applications#2287
NaviaSamal wants to merge 5 commits intogithub:mainfrom
NaviaSamal:feat/add-threat-model-skill

Conversation

@NaviaSamal
Copy link
Copy Markdown

Description

What does this PR build

A Spec-Kit skill (speckit-threatmodel) that performs security threat analysis on LLM interaction surfaces — skills, prompts, templates, hooks, and memory files. (Existing Claude skills created and new ones added)

What does success Looks Like

Running /speckit-threat-model all skills produces:

  1. A FEATURE_DIR/threat-model.md with categorized threats, risk ratings, and mitigations
  2. A FEATURE_DIR/checklists/security-llm.md with actionable verification items
  3. Clear identification of any blocking threats (Critical + High likelihood)

This PR adds a new /speckit.threatmodel command to Spec Kit that performs comprehensive security analysis of Spec-Driven Development projects using the OWASP LLM Top 10 2025 framework. The command analyzes skills, templates, and memory files for potential security vulnerabilities specific to large language model workflows, generating a threat-model.md report and security checklist.

Why This Is Needed

As AI/LLM tools become central to development workflows, security risks specific to these technologies (prompt injection, data poisoning, etc.) need proactive identification. This command integrates security analysis directly into the Spec-Driven Development process, ensuring that threat modeling happens early and continuously rather than as an afterthought. SDD emphasizes that security vulnerabilities discovered in production should feed back into specifications as constraints for future generations. This command brings that feedback loop forward — proactively identifying LLM-specific threats during the specification phase rather than waiting for production incidents.

Testing

  • [ yes] Tested locally with uv run specify --help
  • [ yes] Ran existing tests with uv sync && uv run pytest
  • [No] Tested with a sample project (if applicable)

Manual test results

Agent: GitHub Copilot in VS Code | OS/Shell: macOS/zsh

Command tested Notes
uv run specify init /tmp/speckit-test --ai claude --offline Pass - project scaffolded successfully
uv run specify init /tmp/speckit-test-copilot --integration copilot --offline Pass - project scaffolded successfully and agant available in agent dropdown in copilot chat
/speckit-specify Add a hello world REST endpoint Pass - spec created successfully
/speckit-threatmodel Pass - threat-model.md created under feature directory and security-llm.md file created under checklists
uv run python -m pytest tests/test_agent_config_consistency.py -q 24 passed - Agent configuration and wiring consistency

###Identify affected commands - reporting template per CONTRIBUTING.md

Test selection reasoning

Changed file Affects Test Why
src/specify_cli/integrations/claude/__init__.py specify init T1 Adds "threatmodel" to ARGUMENT_HINTS dict — affects scaffolding of the Claude integration; specify init must produce the new skill correctly
templates/commands/threatmodel.md /speckit.threatmodel T2 New command template — defines the /speckit.threatmodel slash command
tests/integrations/test_integration_base_markdown.py (test infrastructure) Test file only; Added "threatmodel" to COMMAND_STEMS list
tests/integrations/test_integration_base_skills.py (test infrastructure) Test file only ; Added Added "threatmodel" to expected_commands and Added "threatmodel" to _SKILL_COMMANDS list
tests/integrations/test_integration_base_toml.py (test infrastructure) Test file only ; Added "threatmodel" to the command list
tests/integrations/test_integration_base_yaml.py (test infrastructure) Test file only
tests/integrations/test_integration_copilot.py (test infrastructure) Test file only ; Added "threatmodel" to expected_commands and Added speckit.threatmodel.agent.md and speckit.threatmodel.prompt.md to expected file lists
tests/integrations/test_integration_generic.py (test infrastructure) Test file only

Required tests

Prerequisites first:

  • T1: /speckit.specify — The Claude integration change (ARGUMENT_HINTS addition) affects specify init scaffolding. Run specify init with --ai claude and verify the threatmodel skill is created with the correct argument-hint in frontmatter.
  • T2: /speckit.threatmodel — New command template. After scaffolding (T1), invoke /speckit.threatmodel in the agent and verify it runs check-prerequisites.sh, inventories artifacts, and produces threat-model.md output as expected.

AI Disclosure

  • I did not use AI assistance for this contribution
  • [ yes] I did use AI assistance (describe below)

This PR was developed with assistance from GitHub Copilot (in VS Code).

How AI was used

I followed the Spec-Driven Development (SDD) workflow and followed the specify skill format manually, authoring the requirements myself and using GitHub Copilot to help generate and refine artifacts at each stage:

  1. Spec — I wrote the initial feature specification ([spec.md]defining the objective, assumptions, user scenarios, and acceptance criteria.
  2. Plan — I used Copilot to produce the phased implementation plan ([plan.md]from the spec, covering Foundation → Scanning → Analysis → Output.
  3. Tasks — Next I took Copilot's help to break the plan into tracked tasks (tasks.md) with acceptance criteria and verify steps.
  4. Implementation — Copilot assisted in generating the command template (templates/commands/threatmodel.md) and wiring it into the integration layer and test suites.

The OWASP LLM Top 10 2025 threat categories and methodology were sourced from the OWASP Top 10 for LLM Applications 2025 and adapted for Spec-Kit's agent artifact context (skills, templates, memory files).

@NaviaSamal NaviaSamal requested a review from mnriem as a code owner April 21, 2026 00:49
@mnriem
Copy link
Copy Markdown
Collaborator

mnriem commented Apr 21, 2026

Please deliver this as a Spec Kit extension as per https://github.com/github/spec-kit/tree/main/extensions and make sure to host it on your own GitHub repository.

Copy link
Copy Markdown
Collaborator

@mnriem mnriem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comment above

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants