How to Audit an Agent Skill Before Installing It
Inspect the whole package, its sources and its data flow before making it available to an agent.
Open the related free tool →
Treat a skill as a package with behavior
A useful skill can contain instructions, scripts, references, templates and tool dependencies. Reading only its title or summary does not establish what happens when it runs. Start with an exact release or commit and keep the files you reviewed together.
The official skills documentation describes a SKILL.md manifest and supporting resources. Optional metadata can declare tool dependencies and invocation behavior. A package audit should therefore include the referenced files and configuration, not just the visible description. This guide adds a practical manual review process; it is not a malware certification.
1. Establish origin and permission to use it
- Identify the maintainer and canonical repository or distribution. Do not rely on a reposted archive with no clear origin.
- Record the exact version or commit and compare changes before accepting an update.
- Read the license and any separate licenses for bundled code, models, assets or examples.
- Look for dependency pins, release notes and a way to report problems. Popularity alone does not prove trustworthy behavior.
2. Read the manifest as an action plan
Find when the skill triggers, what inputs it expects, what it creates and what it says to do on failure. A narrow description should agree with the full instructions. Trace every reference that can affect an action: scripts, configuration files, install steps, remote endpoints and tool dependencies.
Mark instructions that try to override user intent, conceal an action, request unrelated credentials or make an unsupported success claim. Quoted examples and warnings can contain the same words as dangerous instructions, so interpret them in context. A keyword match is a lead for review, not a conclusion.
3. Trace the package boundary
| Behavior | Questions to answer |
|---|---|
| Reads | Which files, environment variables, credentials and documents can it read? Are they necessary? |
| Writes | Which exact paths or records can change? Is there a recoverable rollback? |
| Executes | What scripts, subprocesses, install hooks or dynamically decoded code can run? |
| Downloads | Are sources pinned and identifiable? Is remote code inspected before execution? |
| Sends | Which destination receives what data, and is that transfer authorized? |
| Persists | Does it add a scheduled job, startup item, installed dependency or saved connection? |
Follow imports and dependency behavior where they matter. A harmless-looking launcher can call a script that performs the significant action. A remote file can change after your review, which is a reason to use an identified version rather than an unreviewed moving target.
4. Use a text checker to organize the review
Open the AI Skill Safety Checker. Paste the manifest and select the supporting text files you want examined. It reports configured patterns with file names, line numbers and evidence excerpts. It does not execute them, unpack archives or fetch links.
Inspect each relevant match in the source. Also review areas with no matches: obfuscated code, binary dependencies, indirect calls and omitted files are outside the checker’s visibility. A comment saying never run a destructive command can still be flagged. Keep the exported report as a review note, not a safety approval.
5. Test only the version and access you intend to use
After review, use disposable data and a suitable restricted environment for a small first test. Define the expected output and the paths or external records that may change. Watch actual behavior and compare the result with the instructions. Do not use real credentials just to see whether an unknown package works.
For a Codex custom agent, review the parent permissions separately; a no-edit instruction does not create a sandbox. The configuration builder can help prepare a bounded specialist file, while the subagent article explains how to assess its scope.
6. Record a decision and recheck updates
Choose accept for a defined use, revise, or reject. Write the exact version, reviewed files, test data, observed effects and unresolved concerns. If you cannot explain where sensitive data goes or what a script changes, do not treat a clean pattern report as permission to proceed.
Review changed files before an update replaces the version you approved. Keep useful evidence and remove access that is no longer needed through the product’s normal controls. A small, understandable skill with a clear purpose is easier to maintain than a large bundle installed only because it contains many features.
Frequently asked questions
Is SKILL.md enough to audit a package?
No. Review referenced scripts, dependencies, metadata and any remote behavior that can affect execution.
Can the checker tell me whether code is malware?
No. It reports local text patterns and has false positives and blind spots. Manual package review remains necessary.
Should I test with real private data?
Begin with disposable data and suitable permissions. Do not expose secrets merely to test an unknown package.
Continue with a practical next step
Created by AI Craft Pad. Contact or report an issue. Check the current generator controls and review the output before use.