feat: add 3 agent security skills (MCP audit, OWASP compliance, supply chain)#1248
feat: add 3 agent security skills (MCP audit, OWASP compliance, supply chain)#1248imran-siddique wants to merge 2 commits intogithub:stagedfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds three new security/governance-focused Agent Skills to expand the repository’s coverage of agent security operations: MCP configuration auditing, OWASP ASI-oriented compliance checking, and agent/plugin supply-chain integrity verification.
Changes:
- Added a new
mcp-security-auditskill describing checks for secrets, shell-injection patterns, and unpinned MCP server dependencies. - Added a new
agent-owasp-complianceskill outlining an OWASP ASI Top 10-oriented assessment workflow and report format. - Added a new
agent-supply-chainskill with integrity manifest generation/verification patterns and CI gating examples.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 9 comments.
| File | Description |
|---|---|
| skills/mcp-security-audit/SKILL.md | New skill documentation for auditing .mcp.json server configs for secrets, injection patterns, and version pinning issues. |
| skills/agent-owasp-compliance/SKILL.md | New skill documentation for assessing agent systems against OWASP ASI Top 10 risks and producing a compliance report. |
| skills/agent-supply-chain/SKILL.md | New skill documentation for hashing-based integrity manifests, tamper detection, dependency pinning audits, and CI verification patterns. |
| description: | | ||
| Audit MCP (Model Context Protocol) server configurations for security issues. Use this skill when: | ||
| - Reviewing .mcp.json files for security risks | ||
| - Checking MCP server args for hardcoded secrets or shell injection patterns | ||
| - Validating that MCP servers use pinned versions (not @latest) | ||
| - Detecting unpinned dependencies in MCP server configurations | ||
| - Auditing which MCP servers a project registers and whether they're on an approved list | ||
| - Checking for environment variable usage vs. hardcoded credentials in MCP configs | ||
| - Any request like "is my MCP config secure?", "audit my MCP servers", or "check .mcp.json" |
There was a problem hiding this comment.
These new skills should be added to the skills index in docs/README.skills.md so they show up in the repo’s documented skill catalog/discovery list (add entries for mcp-security-audit, agent-owasp-compliance, and agent-supply-chain).
| description: | | |
| Audit MCP (Model Context Protocol) server configurations for security issues. Use this skill when: | |
| - Reviewing .mcp.json files for security risks | |
| - Checking MCP server args for hardcoded secrets or shell injection patterns | |
| - Validating that MCP servers use pinned versions (not @latest) | |
| - Detecting unpinned dependencies in MCP server configurations | |
| - Auditing which MCP servers a project registers and whether they're on an approved list | |
| - Checking for environment variable usage vs. hardcoded credentials in MCP configs | |
| - Any request like "is my MCP config secure?", "audit my MCP servers", or "check .mcp.json" | |
| description: 'audit mcp server configurations in .mcp.json files for security issues including secrets exposure, shell injection, unpinned dependencies, and unapproved servers' |
| findings = [] | ||
| args_text = json.dumps(server_config.get("args", [])) | ||
| for pattern, description in DANGEROUS_PATTERNS: | ||
| if re.search(pattern, args_text): | ||
| findings.append({ |
There was a problem hiding this comment.
This snippet calls json.dumps(...) and re.search(...) but this code block doesn’t include imports for json/re (they’re only present in a previous snippet). Add the imports here or explicitly note that this block depends on earlier imports so it’s copy/pasteable.
|
|
||
| Flag MCP servers using `@latest` or unversioned packages. | ||
|
|
||
| ```python | ||
| def check_pinned_versions(server_config: dict) -> list[dict]: |
There was a problem hiding this comment.
The text says this check flags "unversioned packages", but the shown implementation only flags @latest (and otherwise doesn’t emit findings). Either implement an unversioned/unpinned package detection or adjust the wording to match what the code actually checks.
| continue | ||
| findings = [] | ||
| findings.extend(check_secrets(config)) | ||
| findings.extend(check_shell_injection(server_config)) | ||
| findings.extend(check_pinned_versions(server_config)) |
There was a problem hiding this comment.
check_secrets(config) is executed inside the per-server loop, which will duplicate the same secret findings for every server and can misattribute them. Run the secret scan once outside the loop and/or scope it to the specific server config being evaluated.
| def verify_manifest(plugin_dir: str) -> tuple[bool, list[str]]: | ||
| """Verify plugin files against INTEGRITY.json.""" | ||
| root = Path(plugin_dir) | ||
| manifest_path = root / "INTEGRITY.json" | ||
|
|
There was a problem hiding this comment.
The verify_manifest snippet relies on Path, json, hash_file, and generate_manifest, but this code block doesn’t show the needed imports/definitions. Consider adding minimal imports here or adding a brief note that it depends on the previous snippet so readers don’t copy an incomplete block.
skills/agent-supply-chain/SKILL.md
Outdated
| required = ["README.md", ".claude-plugin/plugin.json"] | ||
| missing = [f for f in required if not (root / f).exists()] |
There was a problem hiding this comment.
promotion_check hard-codes .claude-plugin/plugin.json as a required file. If this is meant to apply to this repo’s plugin layout, the canonical path is .github/plugin/plugin.json (see e.g. plugins/awesome-copilot/.github/plugin/plugin.json). Otherwise, please clarify that this required-files list is specific to Claude-style plugins so it’s not misleading.
| required = ["README.md", ".claude-plugin/plugin.json"] | |
| missing = [f for f in required if not (root / f).exists()] | |
| # README is always required; plugin manifest can be in either canonical or legacy location | |
| required = ["README.md"] | |
| missing = [f for f in required if not (root / f).exists()] | |
| # Require at least one plugin manifest in the repo's canonical layout or the legacy Claude layout | |
| plugin_manifest_paths = [ | |
| root / ".github/plugin/plugin.json", | |
| root / ".claude-plugin/plugin.json", | |
| ] | |
| if not any(p.exists() for p in plugin_manifest_paths): | |
| missing.append(".github/plugin/plugin.json (or .claude-plugin/plugin.json)") |
| for c in iter(lambda: f.read(8192), b''): | ||
| h.update(c) | ||
| return h.hexdigest() | ||
|
|
||
| manifest = json.loads(Path('INTEGRITY.json').read_text()) |
There was a problem hiding this comment.
The CI example reads INTEGRITY.json from the current working directory. Unless the workflow cds into the plugin directory first, this will fail for manifests stored under a plugin subfolder (as shown earlier). Consider updating the snippet to cd into the plugin dir or to reference the manifest path explicitly.
| Codebase → Scan for each ASI control: | ||
| ASI-01: Prompt Injection Protection | ||
| ASI-02: Tool Use Governance | ||
| ASI-03: Agency Boundaries | ||
| ASI-04: Escalation Controls |
There was a problem hiding this comment.
The overview implies there are concrete checks for each ASI-01..ASI-10 control, but the document only defines detailed check sections for a subset (01, 02, 05, 07, 09). Either add guidance/check sections for the remaining risks or make it explicit that only partial checks are included so expectations match.
| "risk": "ASI-01", | ||
| "name": "Prompt Injection", | ||
| "status": "pass" if positive_found and not negative_found else "fail", | ||
| "controls_found": positive_matches, | ||
| "vulnerabilities": negative_matches, |
There was a problem hiding this comment.
In this code snippet, positive_found, negative_found, positive_matches, and negative_matches are undefined, so the example as written isn’t runnable. If this is meant as pseudocode, call that out explicitly; otherwise, include a minimal implementation sketch for computing these values (e.g., via grep/ripgrep or AST scanning).
aaronpowell
left a comment
There was a problem hiding this comment.
It looks like you've incorrectly branched from the main branch not staged, and as a result all the materialised plugins are included in this PR.
You can attempt to fix this with a rebase:
git fetch origin staged
git rebase --onto origin/staged origin/main <branch name>
git push --force-with-lease
If that does not resolve it, you can run npm run plugin:clean which will delete the materialised plugins and you can commit that change.
37d2a45 to
eb1b932
Compare
🔍 Skill Validator Results4 resource(s) checked | ✅ All checks passed Full output |
…y chain) - mcp-security-audit: Audit .mcp.json files for hardcoded secrets, shell injection, unpinned versions, dangerous command patterns - agent-owasp-compliance: Check agent systems against OWASP ASI 2026 Top 10 risks with compliance report generation - agent-supply-chain: SHA-256 integrity manifests, tamper detection, version pinning audit, promotion gates for agent plugins Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1. Added 3 new skills to docs/README.skills.md index 2. Added imports (json, re) to shell injection check snippet 3. Updated unpinned deps wording to match code behavior (@latest only) 4. Moved check_secrets() outside per-server loop to avoid duplicates 5. Added imports note to verify_manifest snippet 6. Updated promotion_check to support both .github/plugin and .claude-plugin layouts 7. Updated CI example to cd into plugin directory before verifying 8. Added check sections for all 10 ASI controls (was missing 03, 04, 06, 08, 10) 9. Made ASI-01 code snippet runnable with actual file scanning implementation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
eb1b932 to
d5052e1
Compare
Adds 3 new skills for AI agent security and governance — an area not currently covered in the awesome-copilot collection.
New Skills
1. mcp-security-audit
Audit MCP server configurations for security issues:
2. agent-owasp-compliance
Check agent systems against the OWASP Agentic Security Initiative (ASI) Top 10:
3. agent-supply-chain
Supply chain integrity for agent plugins and tools:
Why These Skills
The existing \�gent-governance\ skill covers governance patterns. These 3 skills extend into specific operational areas:
Related: Agent Governance Toolkit