Skip to content

Commit d5052e1

Browse files
fix: address all 9 review comments
1. Added 3 new skills to docs/README.skills.md index 2. Added imports (json, re) to shell injection check snippet 3. Updated unpinned deps wording to match code behavior (@latest only) 4. Moved check_secrets() outside per-server loop to avoid duplicates 5. Added imports note to verify_manifest snippet 6. Updated promotion_check to support both .github/plugin and .claude-plugin layouts 7. Updated CI example to cd into plugin directory before verifying 8. Added check sections for all 10 ASI controls (was missing 03, 04, 06, 08, 10) 9. Made ASI-01 code snippet runnable with actual file scanning implementation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 45ce2fa commit d5052e1

File tree

4 files changed

+132
-18
lines changed

4 files changed

+132
-18
lines changed

docs/README.skills.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,8 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#adding-skills) for guidelines on how to
2828
| ---- | ----------- | -------------- |
2929
| [add-educational-comments](../skills/add-educational-comments/SKILL.md) | Add educational comments to the file specified, or prompt asking for file to comment if one is not provided. | None |
3030
| [agent-governance](../skills/agent-governance/SKILL.md) | Patterns and techniques for adding governance, safety, and trust controls to AI agent systems. Use this skill when:<br />- Building AI agents that call external tools (APIs, databases, file systems)<br />- Implementing policy-based access controls for agent tool usage<br />- Adding semantic intent classification to detect dangerous prompts<br />- Creating trust scoring systems for multi-agent workflows<br />- Building audit trails for agent actions and decisions<br />- Enforcing rate limits, content filters, or tool restrictions on agents<br />- Working with any agent framework (PydanticAI, CrewAI, OpenAI Agents, LangChain, AutoGen) | None |
31+
| [agent-owasp-compliance](../skills/agent-owasp-compliance/SKILL.md) | Check AI agent systems against the OWASP Agentic Security Initiative (ASI) Top 10 risks. Evaluates all 10 controls (prompt injection, tool governance, excessive agency, escalation, trust boundaries, logging, identity, policy integrity, supply chain, behavioral anomaly) and generates an X/10 compliance report. | None |
32+
| [agent-supply-chain](../skills/agent-supply-chain/SKILL.md) | Verify supply chain integrity for AI agent plugins and tools. Generate SHA-256 integrity manifests, verify installed plugins match published manifests, detect tampered files, audit dependency version pinning, and gate plugin promotion from dev to production. | None |
3133
| [agentic-eval](../skills/agentic-eval/SKILL.md) | Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when:<br />- Implementing self-critique and reflection loops<br />- Building evaluator-optimizer pipelines for quality-critical generation<br />- Creating test-driven code refinement workflows<br />- Designing rubric-based or LLM-as-judge evaluation systems<br />- Adding iterative improvement to agent outputs (code, reports, analysis)<br />- Measuring and improving agent response quality | None |
3234
| [ai-prompt-engineering-safety-review](../skills/ai-prompt-engineering-safety-review/SKILL.md) | Comprehensive AI prompt engineering safety review and improvement prompt. Analyzes prompts for safety, bias, security vulnerabilities, and effectiveness while providing detailed improvement recommendations with extensive frameworks, testing methodologies, and educational content. | None |
3335
| [appinsights-instrumentation](../skills/appinsights-instrumentation/SKILL.md) | Instrument a webapp to send useful telemetry data to Azure App Insights | `LICENSE.txt`<br />`examples`<br />`references/ASPNETCORE.md`<br />`references/AUTO.md`<br />`references/NODEJS.md`<br />`references/PYTHON.md`<br />`scripts/appinsights.ps1` |
@@ -183,6 +185,7 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#adding-skills) for guidelines on how to
183185
| [markdown-to-html](../skills/markdown-to-html/SKILL.md) | Convert Markdown files to HTML similar to `marked.js`, `pandoc`, `gomarkdown/markdown`, or similar tools; or writing custom script to convert markdown to html and/or working on web template systems like `jekyll/jekyll`, `gohugoio/hugo`, or similar web templating systems that utilize markdown documents, converting them to html. Use when asked to "convert markdown to html", "transform md to html", "render markdown", "generate html from markdown", or when working with .md files and/or web a templating system that converts markdown to HTML output. Supports CLI and Node.js workflows with GFM, CommonMark, and standard Markdown flavors. | `references/basic-markdown-to-html.md`<br />`references/basic-markdown.md`<br />`references/code-blocks-to-html.md`<br />`references/code-blocks.md`<br />`references/collapsed-sections-to-html.md`<br />`references/collapsed-sections.md`<br />`references/gomarkdown.md`<br />`references/hugo.md`<br />`references/jekyll.md`<br />`references/marked.md`<br />`references/pandoc.md`<br />`references/tables-to-html.md`<br />`references/tables.md`<br />`references/writing-mathematical-expressions-to-html.md`<br />`references/writing-mathematical-expressions.md` |
184186
| [mcp-cli](../skills/mcp-cli/SKILL.md) | Interface for MCP (Model Context Protocol) servers via CLI. Use when you need to interact with external tools, APIs, or data sources through MCP servers, list available MCP servers/tools, or call MCP tools from command line. | None |
185187
| [mcp-copilot-studio-server-generator](../skills/mcp-copilot-studio-server-generator/SKILL.md) | Generate a complete MCP server implementation optimized for Copilot Studio integration with proper schema constraints and streamable HTTP support | None |
188+
| [mcp-security-audit](../skills/mcp-security-audit/SKILL.md) | Audit MCP server configurations in .mcp.json files for security issues including secrets exposure, shell injection patterns, unpinned dependencies, and dangerous command patterns. | None |
186189
| [mcp-create-adaptive-cards](../skills/mcp-create-adaptive-cards/SKILL.md) | Skill converted from mcp-create-adaptive-cards.prompt.md | None |
187190
| [mcp-create-declarative-agent](../skills/mcp-create-declarative-agent/SKILL.md) | Skill converted from mcp-create-declarative-agent.prompt.md | None |
188191
| [mcp-deploy-manage-agents](../skills/mcp-deploy-manage-agents/SKILL.md) | Skill converted from mcp-deploy-manage-agents.prompt.md | None |

skills/agent-owasp-compliance/SKILL.md

Lines changed: 103 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -56,22 +56,38 @@ Codebase → Scan for each ASI control:
5656
Look for input validation that runs **before** tool execution, not after LLM generation.
5757

5858
```python
59+
import re
60+
from pathlib import Path
61+
5962
def check_asi_01(project_path: str) -> dict:
6063
"""ASI-01: Is user input validated before reaching tool execution?"""
61-
signals = {
62-
"positive": [
63-
"input_validation", "validate_input", "sanitize",
64-
"classify_intent", "prompt_injection", "threat_detect",
65-
"PolicyEvaluator", "PolicyEngine", "check_content",
66-
],
67-
"negative": [
68-
"eval(", "exec(", "subprocess.run(.*shell=True",
69-
"os.system(", "input()", # raw input passed to tools
70-
]
71-
}
72-
# Search codebase for these patterns
73-
# Positive signals = controls exist
74-
# Negative signals = potential vulnerabilities
64+
positive_patterns = [
65+
"input_validation", "validate_input", "sanitize",
66+
"classify_intent", "prompt_injection", "threat_detect",
67+
"PolicyEvaluator", "PolicyEngine", "check_content",
68+
]
69+
negative_patterns = [
70+
r"eval\(", r"exec\(", r"subprocess\.run\(.*shell=True",
71+
r"os\.system\(",
72+
]
73+
74+
# Scan Python files for signals
75+
root = Path(project_path)
76+
positive_matches = []
77+
negative_matches = []
78+
79+
for py_file in root.rglob("*.py"):
80+
content = py_file.read_text(errors="ignore")
81+
for pattern in positive_patterns:
82+
if pattern in content:
83+
positive_matches.append(f"{py_file.name}: {pattern}")
84+
for pattern in negative_patterns:
85+
if re.search(pattern, content):
86+
negative_matches.append(f"{py_file.name}: {pattern}")
87+
88+
positive_found = len(positive_matches) > 0
89+
negative_found = len(negative_matches) > 0
90+
7591
return {
7692
"risk": "ASI-01",
7793
"name": "Prompt Injection",
@@ -122,6 +138,34 @@ def execute_tool(name: str, args: dict):
122138

123139
---
124140

141+
## Check ASI-03: Excessive Agency
142+
143+
Verify agent capabilities are bounded — not open-ended.
144+
145+
**What to search for:**
146+
- Explicit capability lists or execution rings
147+
- Scope limits on what the agent can access
148+
- Principle of least privilege applied to tool access
149+
150+
**Failing:** Agent has access to all tools by default.
151+
**Passing:** Agent capabilities defined as a fixed allowlist, unknown tools denied.
152+
153+
---
154+
155+
## Check ASI-04: Unauthorized Escalation
156+
157+
Verify agents cannot promote their own privileges.
158+
159+
**What to search for:**
160+
- Privilege level checks before sensitive operations
161+
- No self-promotion patterns (agent changing its own trust score or role)
162+
- Escalation requires external attestation (human or SRE witness)
163+
164+
**Failing:** Agent can modify its own configuration or permissions.
165+
**Passing:** Privilege changes require out-of-band approval (e.g., Ring 0 requires SRE attestation).
166+
167+
---
168+
125169
## Check ASI-05: Trust Boundary Violation
126170

127171
In multi-agent systems, verify that agents verify each other's identity before accepting instructions.
@@ -145,6 +189,21 @@ def accept_task(sender_id: str, task: dict):
145189

146190
---
147191

192+
## Check ASI-06: Insufficient Logging
193+
194+
Verify all agent actions produce structured, tamper-evident audit entries.
195+
196+
**What to search for:**
197+
- Structured logging for every tool call (not just print statements)
198+
- Audit entries include: timestamp, agent ID, tool name, args, result, policy decision
199+
- Append-only or hash-chained log format
200+
- Logs stored separately from agent-writable directories
201+
202+
**Failing:** Agent actions logged via `print()` or not logged at all.
203+
**Passing:** Structured JSONL audit trail with chain hashes, exported to secure storage.
204+
205+
---
206+
148207
## Check ASI-07: Insecure Identity
149208

150209
Verify agents have cryptographic identity, not just string names.
@@ -162,6 +221,21 @@ Verify agents have cryptographic identity, not just string names.
162221

163222
---
164223

224+
## Check ASI-08: Policy Bypass
225+
226+
Verify policy enforcement is deterministic — not LLM-based.
227+
228+
**What to search for:**
229+
- Policy evaluation uses deterministic logic (YAML rules, code predicates)
230+
- No LLM calls in the enforcement path
231+
- Policy checks cannot be skipped or overridden by the agent
232+
- Fail-closed behavior (if policy check errors, action is denied)
233+
234+
**Failing:** Agent decides its own permissions via prompt ("Am I allowed to...?").
235+
**Passing:** PolicyEvaluator.evaluate() returns allow/deny in <0.1ms, no LLM involved.
236+
237+
---
238+
165239
## Check ASI-09: Supply Chain Integrity
166240

167241
Verify agent plugins and tools have integrity verification.
@@ -174,6 +248,21 @@ Verify agent plugins and tools have integrity verification.
174248

175249
---
176250

251+
## Check ASI-10: Behavioral Anomaly
252+
253+
Verify the system can detect and respond to agent behavioral drift.
254+
255+
**What to search for:**
256+
- Circuit breakers that trip on repeated failures
257+
- Trust score decay over time (temporal decay)
258+
- Kill switch or emergency stop capability
259+
- Anomaly detection on tool call patterns (frequency, targets, timing)
260+
261+
**Failing:** No mechanism to stop a misbehaving agent automatically.
262+
**Passing:** Circuit breaker trips after N failures, trust decays without activity, kill switch available.
263+
264+
---
265+
177266
## Compliance Report Format
178267

179268
```markdown

skills/agent-supply-chain/SKILL.md

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,10 @@ print(f"Generated manifest: {manifest['file_count']} files, "
120120
Check that current files match the manifest.
121121

122122
```python
123+
# Requires: hash_file() and generate_manifest() from Pattern 1 above
124+
import json
125+
from pathlib import Path
126+
123127
def verify_manifest(plugin_dir: str) -> tuple[bool, list[str]]:
124128
"""Verify plugin files against INTEGRITY.json."""
125129
root = Path(plugin_dir)
@@ -229,8 +233,17 @@ def promotion_check(plugin_dir: str) -> dict:
229233

230234
# 2. Required files exist
231235
root = Path(plugin_dir)
232-
required = ["README.md", ".claude-plugin/plugin.json"]
236+
required = ["README.md"]
233237
missing = [f for f in required if not (root / f).exists()]
238+
239+
# Require at least one plugin manifest (supports both layouts)
240+
manifest_paths = [
241+
root / ".github/plugin/plugin.json",
242+
root / ".claude-plugin/plugin.json",
243+
]
244+
if not any(p.exists() for p in manifest_paths):
245+
missing.append(".github/plugin/plugin.json (or .claude-plugin/plugin.json)")
246+
234247
checks["required_files"] = {
235248
"passed": len(missing) == 0,
236249
"missing": missing
@@ -274,6 +287,8 @@ Add to your GitHub Actions workflow:
274287
```yaml
275288
- name: Verify plugin integrity
276289
run: |
290+
PLUGIN_DIR="${{ matrix.plugin || '.' }}"
291+
cd "$PLUGIN_DIR"
277292
python -c "
278293
from pathlib import Path
279294
import json, hashlib, sys

skills/mcp-security-audit/SKILL.md

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ description: |
99
- Auditing which MCP servers a project registers and whether they're on an approved list
1010
- Checking for environment variable usage vs. hardcoded credentials in MCP configs
1111
- Any request like "is my MCP config secure?", "audit my MCP servers", or "check .mcp.json"
12+
keywords: [mcp, security, audit, secrets, shell-injection, supply-chain, governance]
1213
---
1314

1415
# MCP Security Audit
@@ -112,6 +113,9 @@ def check_secrets(mcp_config: dict) -> list[dict]:
112113
Detect dangerous command patterns in MCP server args.
113114

114115
```python
116+
import json
117+
import re
118+
115119
DANGEROUS_PATTERNS = [
116120
(r'\$\(', "Command substitution $(...)"),
117121
(r'`[^`]+`', "Backtick command substitution"),
@@ -145,11 +149,11 @@ def check_shell_injection(server_config: dict) -> list[dict]:
145149

146150
## Audit Check 3: Unpinned Dependencies
147151

148-
Flag MCP servers using `@latest` or unversioned packages.
152+
Flag MCP servers using `@latest` in their package references.
149153

150154
```python
151155
def check_pinned_versions(server_config: dict) -> list[dict]:
152-
"""Check that MCP server dependencies use pinned versions."""
156+
"""Check that MCP server dependencies use pinned versions, not @latest."""
153157
findings = []
154158
args = server_config.get("args", [])
155159
for arg in args:
@@ -204,11 +208,14 @@ def audit_mcp_config(mcp_path: str) -> dict:
204208
results = {"file": str(path), "servers": {}, "summary": {}}
205209
total_findings = []
206210

211+
# Run secrets check once on the whole config (not per-server)
212+
config_level_findings = check_secrets(config)
213+
total_findings.extend(config_level_findings)
214+
207215
for name, server_config in servers.items():
208216
if not isinstance(server_config, dict):
209217
continue
210218
findings = []
211-
findings.extend(check_secrets(config))
212219
findings.extend(check_shell_injection(server_config))
213220
findings.extend(check_pinned_versions(server_config))
214221
results["servers"][name] = {

0 commit comments

Comments
 (0)