A Marketplace Listing Is Not Install Evidence: The Agent Package Passport

An agent marketplace is useful for discovering a product, but a polished listing does not establish what will execute in your workspace, which data it can reach, or how it will change after installation. Before connecting an agent to business systems, ask for evidence tied to the exact package version and the exact permissions you plan to grant. The marketplace catalogue can help a buyer compare use cases and shortlist candidates; treat each card as a starting point for verification, not as a security certification or procurement approval.

The practical recommendation is to create a short, version-bound package passport before install. It records who owns the package, what exact artifact is being installed, what capabilities and credentials it requires, where its data can move, how it is isolated, how changes are reviewed, who operates it, and how to stop and remove it. Then make one of three decisions: go for a tightly bounded use with evidence for every hard gate; pilot only in a disposable or synthetic environment with no consequential write access; or block when identity, behavior, egress, update control, or exit cannot be demonstrated. A listing’s popularity, stars, testimonials, or vendor-authored score never overrides a failed hard gate.

This is a buyer protocol, not a claim that every marketplace agent is unsafe. The same discipline applies to a skill, connector, hosted bot, MCP server, packaged workflow, or downloadable agent. The higher the autonomy and business consequence, the more evidence and runtime control the buyer should require.

The gap between discovery and an install decision

A catalog answers a discovery question: what might do this job? A listing may describe a use case, show integrations, quote a performance outcome, display a publisher name, and invite a demo. Those details can help form a shortlist. They do not by themselves answer the deployment questions that decide whether the artifact can safely enter a real environment.

The distinction matters because “agent” covers different delivery models. A package may be a local set of instructions and scripts, a hosted service called through an API, a connector that receives delegated credentials, a workflow imported into an automation platform, or a live source that continues to change after installation. The package may be only one component in a chain that also includes a model provider, a tool server, an identity provider, a data store, and a publisher-controlled update channel. Each boundary can change what is actually being trusted.

The source marketplace page, for example, makes broad catalogue-level claims about agents being compared, verified, and ready to integrate. Its page also contains individual outcome figures. Those are claims presented by the site itself; this article does not independently validate the count, user metrics, satisfaction rate, time-to-deployment estimate, or advertised agent outcomes. A buyer should ask what “verified” means for a particular listing, which version was checked, what test was performed, and whether the evidence is available for inspection. A badge without scope, date, artifact identity, and limitations is not enough to reproduce a decision.

A recurring community question is whether teams can give a marketplace app different read and write rights for different job roles. One OpenAI developer-community request describes an all-or-nothing role assignment where users who only need to read may inherit a connector that can also modify records. That is one participant’s reported product limitation, not evidence about every platform; it does expose a practical buyer question: can the installed package’s actions be narrowed for each user, or does the marketplace control stop at discoverability? See the permission-level request.

Why a marketplace does not transfer responsibility

A marketplace operator can reduce some search cost by curating, testing, or screening submissions. The buyer still needs to know what was screened, when it was screened, which exact version was inspected, and what the screening could not detect. Static inspection can miss behavior that depends on remote content, credentials, model decisions, runtime configuration, tenant-specific data, or an update that arrives later. Conversely, a listing without a public security badge is not automatically dangerous. Absence of visible evidence means the buyer has uncertainty to manage; it is not proof of malice.

Treat the catalog as a distribution surface in a software supply chain. CISA’s customer-oriented software supply-chain guidance recommends mechanisms to verify the authenticity and integrity of updates, use approved distribution channels, review documented changes, and reassess updates that introduce new functionality or unclear access-control instructions. It also discusses supplier and product artifacts that can help customers assess dependencies. That guidance is written for software customers generally, not specifically for AI agents, so applying it to an agent package is an explicit transfer of the supply-chain principle rather than a CISA agent-marketplace certification. The primary source is CISA’s recommended practices for software customers.

A buyer can also mistake data location for security. An empirical paper in the Journal of Cybersecurity uses a global country-year panel covering 2000–2022 and reports no statistically significant relationship between data-localization policies and several cyber-incident outcomes in its models. Its unit is national policy and country-level incident outcomes, not an individual vendor or agent deployment. It cannot tell you whether a particular agent is secure, compliant, or appropriate. It is a useful counterweight to the inference that “hosted in my region” alone proves security: geography may matter for legal and operational reasons, but it does not substitute for identity controls, package integrity, constrained credentials, monitoring, or incident response. Read the study and its limitations.

The evidence converges on a simple buyer stance: verify the artifact and the operating boundary. It does not support a universal claim that marketplace screening is useless, that every agent requires a full enterprise audit, or that one checklist can eliminate risk. A small, read-only agent operating on public information can be approved with a proportionate review. A package that sends customer messages, changes records, moves money, or handles sensitive files needs stronger evidence and operational controls.

The package passport: ten fields that turn a card into a decision

The passport is an original reusable decision record. It is deliberately evidence-oriented: every field must point to a file, URL, setting, test result, contract clause, owner, or explicit unresolved item. “Vendor says yes” is a recorded claim, not the same thing as proof. Use unknown rather than silently treating a missing field as safe.

The passport is not a vendor questionnaire to send back unanswered. It is a compact mapping between a buyer’s decision and the evidence that makes the decision repeatable. A response may be “not applicable,” but it should say why. For example, a local static prompt bundle may not need a hosted subprocessor map; it still needs source provenance, file and shell behavior review, and a plan for how updates are accepted. A hosted retrieval agent may have no local executable, yet its data-flow map and credential scope become central.

The passport should name the exact package version rather than only the product brand. A vendor can improve a package after review, but the buyer’s approval should not silently expand to future capabilities. A new version, new endpoint, new dependency, permission increase, changed retention term, ownership transfer, or new schedule is a change event. Re-run the affected checks before the new version inherits production access.

Turn the fields into a go, pilot, or block decision

A weighted score is tempting because it produces a single number, but it can hide a fatal gap. A strong publisher reputation must not compensate for an unknown package digest; a good test result must not compensate for an unrestricted token; and a data-processing agreement must not compensate for an unreviewed auto-update path. Use non-compensating gates first, then compare the candidates that pass.

Go means the business task is clear, the exact artifact is identifiable, required capabilities are narrow and justified, data destinations are acceptable, execution is bounded, the owner can observe outcomes, and stop/rollback steps have been rehearsed. For a low-impact read-only task, the evidence can be lightweight if the consequences are genuinely limited. A production write agent should have separate credentials, an action approval boundary where appropriate, an audit record, and a tested recovery path.

Pilot means no hard stop remains, but evidence or operational maturity is incomplete in a way that can be safely contained. Use synthetic or explicitly approved low-sensitivity data, a dedicated environment, a short expiry, a named observer, and a strict action ceiling. A pilot must have a written success measure and a kill condition. Do not connect a production mailbox or unrestricted CRM simply because the first demo worked.

Block means identity is ambiguous; the proposed artifact cannot be pinned; the package’s capabilities do not match the listing; data egress is unknown; a broad credential is mandatory; updates cannot be reviewed; or no one can revoke, disable, and investigate the agent. A block is a decision about the current evidence and use case, not a permanent judgement about the vendor. A supplier can return with a better-defined version and a more constrained deployment.

The decision should state what it does not approve. “Approved for a sandbox test with synthetic leads and read-only access” is operationally meaningful. “Approved vendor” is too vague: it may be misread as approval for a different package, permission set, data class, or future version.

A practical review sequence

Start with the business task, not the agent’s feature list. Write one sentence describing the intended result, the system of record, the user who owns the result, and what the agent is not allowed to do. A task such as “prepare a draft answer from public FAQs for a support employee” is materially different from “resolve customer tickets and issue refunds.” The first might run with read-only access and a human publishing step; the second has customer impact, write permissions, money movement, and exception handling to control.

Next, capture the listing and exact installation target. Record the publisher, version, artifact digest or immutable source revision, date checked, and the path through which the package is installed. If the product is a hosted service with no buyer-visible binary, ask which service and model versions are active, how releases are announced, and whether an organization can pin or delay behavior changes. A commit hash is useful for code, but it does not pin a remote model, prompt, API, feature flag, or data-retention policy. Record those separately.

Then inventory operations from the artifact or platform, not merely the marketing description. For an MCP server, record tools and their descriptions; for a workflow, inspect nodes, credentials, triggers, and destinations; for a hosted agent, review scopes, API access, data-processing terms, and logs; for a local skill, inspect scripts, shell commands, file patterns, network calls, and any instructions that tell the model to take actions. A tool’s label is not its security boundary. A tool called “search” might retrieve sensitive records; a tool called “sync” might write or delete.

Trace credentials from creation to removal. Prefer a dedicated service identity, minimum scope, short-lived credential where available, and secret storage outside the package. The package should never require a secret in public listing text or a checked-in file. Confirm that revocation can happen without vendor assistance, that credential rotation is possible, and that an agent cannot read credentials belonging to unrelated tasks. A permission prompt is useful only if its wording corresponds to enforceable scopes and the platform can preserve the boundary at runtime.

Map data movement in both ordinary and failure cases. Identify what the agent reads, what it sends to a model or tool provider, where logs are stored, how support staff can access traces, how long information remains, and what happens on deletion. Include prompts, tool results, attachments, embeddings, telemetry, and error reports when those can contain sensitive information. Ask whether a human review or support session can expose data to a different audience. Geography is one attribute in this map; the OUP study above is a reminder that location alone is not proof of stronger cybersecurity. Legal transfer requirements still need their own jurisdiction-specific review.

Finally, make an update and exit plan before enabling the agent. Disable silent auto-updates for a high-consequence package unless the platform provides a controlled staged process. Review each release’s code or manifest diff, capability changes, dependencies, and notes. Preserve the last known-good version when technically possible. Test that rollback actually restores the package and its configuration; some hosted systems cannot roll back a model or server-side behavior. Confirm an operator can disable schedules, revoke credentials, stop remote connections, export business records, and delete or retain logs according to policy.

A reusable evidence register and executable checker

The following passport is a starting data shape, not a security standard or certification. It intentionally refuses to infer a “pass” from a product score. Keep source URLs, hashes, screenshots, approvals, and test results in your controlled evidence store; do not paste secrets into the passport. The fields can be adapted for a marketplace item, but the reviewer should preserve a history of changed values so a later audit can answer what was approved at the time.

{ "package": { "listingUrl": "https:\/\/catalog.example/agent/123", "publisher": "Example Vendor Ltd", "version": "2.4.1", "artifactDigest": "sha256:REPLACE_WITH_VERIFIED_DIGEST", "installSource": "approved-registry", "checkedAt": "2026-11-04" }, "task": { "owner": "support-ops@example.invalid", "purpose": "draft answers from public help content", "forbiddenActions": ["send customer reply", "issue refund"] }, "capabilities": ["read:public-help-content"], "dataFlow": { "classes": ["public"], "destinations": ["approved-model-endpoint"], "retentionDays": 0, "deletionEvidence": "ticket-4821" }, "controls": { "sandbox": true, "egressAllowlistReviewed": true, "autoUpdate": false, "rollbackTarget": "2.3.8", "credentialRevocableByOwner": true, "actionLogAvailable": true }, "operations": { "businessOwner": "support-ops@example.invalid", "technicalOwner": "platform@example.invalid", "expiryDate": "2026-12-04", "decision": "pilot" } } 

A small validation program can catch incomplete passports before the review meeting. It cannot establish that a publisher is honest, a digest is authentic, a test is adequate, or a runtime setting is effective. The reviewer must verify the linked evidence, and the deployment platform must enforce the controls. The example below checks structural completeness and a few fail-closed rules. It uses only Node.js built-ins and expects a JSON file path as its first argument.

// passport-check.mjs — Node.js 20+; structural gate, not a security scanner import { readFile } from 'node:fs/promises'; const file = process.argv[2]; if (!file) { console.error('Usage: node passport-check.mjs passport.json'); process.exit(2); } const passport = JSON.parse(await readFile(file, 'utf8')); const errors = []; const need = (condition, message) => { if (!condition) errors.push(message); }; const pkg = passport.package ?? {}; const task = passport.task ?? {}; const flow = passport.dataFlow ?? {}; const controls = passport.controls ?? {}; const ops = passport.operations ?? {}; need((pkg.listingUrl ?? '').startsWith('https:') && (pkg.listingUrl ?? '').includes('://'), 'listingUrl must be HTTPS'); need(Boolean(pkg.publisher), 'publisher is missing'); need(Boolean(pkg.version), 'version is missing'); need(/^sha256:[a-f0-9]{64}$/i.test(pkg.artifactDigest ?? ''), 'artifactDigest must be a verified SHA-256 value'); need(Boolean(task.owner && task.purpose), 'task owner and purpose are required'); need(Array.isArray(passport.capabilities) && passport.capabilities.length > 0, 'capabilities must be an explicit non-empty list'); need(Array.isArray(flow.destinations) && flow.destinations.length > 0, 'data destinations must be explicit'); need(typeof controls.autoUpdate === 'boolean', 'autoUpdate must be explicitly true or false'); need(Boolean(controls.rollbackTarget), 'rollback target is missing'); need(controls.credentialRevocableByOwner === true, 'owner-controlled credential revocation is required'); need(Boolean(ops.businessOwner && ops.technicalOwner && ops.expiryDate), 'named owners and review expiry are required'); need(['go', 'pilot', 'block'].includes(ops.decision), 'decision must be go, pilot, or block'); if (controls.autoUpdate === true && ops.decision === 'go') { errors.push('auto-update requires a separately documented staged review before go'); } if (!controls.sandbox && (passport.capabilities ?? []).some(x => /write|delete|send|pay/i.test(x))) { errors.push('write-capable package has no sandbox or compensating boundary'); } if (errors.length) { console.error('BLOCK: passport is incomplete'); for (const error of errors) console.error(`- ${error}`); process.exit(1); } console.log(`STRUCTURE OK: ${ops.decision} review recorded for ${pkg.publisher} ${pkg.version}`); 

The placeholder digest in the JSON example will fail the checker until replaced by a verified SHA-256 value. Do not generate a hash of a listing page and call it a package hash: hash the actual immutable artifact or source bundle you received, and preserve how it was obtained. For a hosted product, a package digest may not exist. In that case, document the provider’s versioning and release evidence, record the limitation, and require a compensating control such as a staged environment and explicit approval before access expands. Do not fake a digest simply to satisfy a field.

A strong passport distinguishes three different things: what the publisher claims, what the buyer independently verified, and what remains unknown. A practical evidence register can add columns for claim, artifact or control, verifier, date, result, scope, and follow-up owner. If an evaluation used synthetic records, preserve the fixture and the expected outcomes. If a scan covered source code but not runtime network behavior, record that boundary. If no independent review exists, say so. An honest not tested result is more useful than a green symbol without scope.

Measure the real deployment boundary, not just the package

A listing’s declared permissions may be narrower than its effective access. An agent can inherit broad credentials from its host, invoke a shared tool that has more authority than the agent needs, or receive retrieved content that contains hostile instructions. A marketplace may have verified the package but cannot know how your organization configured its CRM token, file shares, network rules, or user roles. The installation review must therefore include the platform and integration path.

Run the first evaluation in a dedicated environment. Start with read-only data, synthetic records, and an explicit denylist for business-changing operations. Observe every tool call and destination. Attempt negative cases: ask the agent to read a forbidden file, send data to an unapproved endpoint, perform a write, continue after its credential is revoked, and act on malicious instructions embedded in a retrieved document. The purpose is not to prove that prompt injection can never happen; the purpose is to verify that a successful manipulation still hits an enforceable boundary.

For actions with consequences, place authorization outside the model’s natural-language reasoning. A user should approve a concrete proposed action with target, amount or fields, expiry, and one-time use where practical. The executor should recheck the authorization and current policy at the moment of action. A general “yes, this agent may use the CRM” is not equivalent to authorization for every future contact, field update, export, or deletion. This article’s package passport is about admission and lifecycle control; it complements a separate per-action authorization protocol rather than replacing it.

Monitor the deployed system for changes in the package version, tool inventory, permission scopes, network destinations, error rates, and action patterns. Define who receives alerts and what conditions automatically pause the agent. Logging needs to be sufficient for investigation but should not collect unnecessary personal data or credentials. Keep logs access-controlled, time-bounded, and covered by your retention policy. A log that records only “agent completed” is rarely enough to reconstruct a disputed action; a log that stores every prompt forever may create a privacy problem of its own.

PwC’s governance discussion recommends clear agent identity and role, task-specific access, auditable activity, ownership, and oversight that grows with autonomy and criticality. That is a consulting perspective rather than a regulator’s binding rule, but it aligns with the passport’s operating fields. PwC also notes that detailed observability may be infeasible, privacy-sensitive, or disproportionately costly in some scenarios. That qualification matters: collect the evidence needed to intervene and investigate, then minimize or protect it. See PwC’s discussion of governance and ongoing oversight.

The update path is part of the product

A buyer often evaluates a demo once and assumes the evaluated behavior is the installed behavior. That assumption fails when the marketplace points to a live repository, when a hosted service changes its prompt or tools server-side, or when an installer follows a moving tag. Capture the point-in-time artifact that was actually evaluated. For source-based packages, use a commit or immutable version and preserve a digest. For hosted services, request release notes, behavior-change notices, model/provider changes, endpoint inventory, and a way to delay a release that changes risk.

Treat a new capability as a fresh permission request. If an update adds a tool that can send messages, reads a new data class, introduces a new subprocess, changes a schedule, or broadens OAuth scopes, it is not merely a patch. Review the diff, re-run relevant negative cases, and compare the new passport with the approved one. CISA’s customer guidance specifically warns that updates can arrive through unreliable sources, expose new functionality during integration, or require unclear changes to allowlists and access policies. The guidance is broad software-supply-chain advice; applying it to a marketplace agent means the buyer should verify origin, integrity, changes, and deployment effects rather than assuming an update is benign.

Rollback also needs a precise meaning. Restoring a local package does not necessarily restore the hosted model, remote tool server, data schema, OAuth consent, or actions already taken. A rollback plan should identify which components can be returned to a known state, which effects require business compensation, and which records must remain for audit. Test disablement and revocation independently: an agent may stop receiving new prompts while a scheduled job or long-running task continues. Ask the provider to explain cancellation semantics and verify them in a safe environment.

A buyer can accept that some parts are not pin-able, but the uncertainty should influence the decision. If the remote service cannot freeze a model or prompt, restrict it to a task where behavior drift has low consequence, keep an approval gate on writes, and increase monitoring. If the supplier cannot explain changes or incidents, do not place sensitive data or consequential actions behind the package. The right result may be to use a stable workflow or a human-assisted tool instead of an autonomous agent.

What the video and community material do—and do not—show

A September 2026 video demonstration described on DutchStartup.ai builds a travel agent connected to a toolkit and shows permissions and scopes. That is a useful design illustration: capabilities should be bounded in the application that executes the work. The page provides a synopsis and links to the video, but I could not verify a full caption transcript in this research pass. I therefore use it only to identify the scope-and-permission topic, not as evidence that a particular marketplace, product, or implementation is safe. The video summary and description are not an independent security audit.

The community request about role-level access is similarly narrow. It illustrates a buyer’s operational concern about granting write capability to people who only need read access. It does not establish how common that limitation is or whether a vendor later changed the product. For a procurement decision, reproduce the issue in the exact tenant and plan you intend to use, and ask the supplier to document the supported controls.

These examples point to a practical information gap. Many guides explain how to find or install an agent, and many security pieces explain a class of attacks. Buyers still need a portable record that binds the listing to the artifact, permission set, data path, update process, owner, and removal procedure. The passport provides that record without pretending that a checklist is a penetration test or that a marketplace listing can certify the buyer’s environment.

Failure modes that invalidate a green-looking approval

Publisher-name collision. A familiar brand name appears beside a package, but the publisher account, domain, signing identity, or support contact is different. Verify the identity chain and the specific maintainer. Similar names are not proof of impersonation, but they are not proof of identity either.

Version drift after review. A reviewer approves one version, while an installer or auto-update channel fetches a later release. Pin or stage the version where possible; otherwise require a notice and review gate before it gains access.

Permission labels that hide operations. A package says “connect to CRM” without listing whether it can create, modify, export, delete, or message. Inspect underlying tool definitions and test the effective scope. If the platform only offers a broad connector, narrow access at the identity or data layer.

Credential inheritance. The agent receives a user token with privileges accumulated over years. The package may be small, but its effective reach is the user’s access. Use a dedicated identity with task-specific scopes and test revocation.

Unknown egress. A tool can send data to an undocumented analytics, model, or logging endpoint. The provider’s region statement may not disclose every destination. Ask for a data-flow map and constrain network routes if the deployment model permits.

A demo that excludes failure cases. Happy-path results do not test prompt injection, malformed inputs, duplicate calls, timeouts, partial completion, and outages. Define a small negative-case fixture around the intended task; do not present a synthetic fixture as production performance.

Rollback that restores only part of the system. The package rolls back while the remote service, credentials, schedule, or changed business records remain. Document each component separately and rehearse compensation for irreversible effects.

No exit route. The user can disable a button but cannot stop background schedules, revoke tokens, retrieve records, or request deletion. Verify each step with the provider and record what evidence will remain.

Logs that are either useless or excessive. Minimal logs may not reveal which record changed; expansive prompt logging may retain sensitive data. Specify event fields, access restrictions, retention, and redaction before the pilot.

A marketplace badge treated as an approval. A badge can mean listing review, identity verification, vulnerability scanning, policy compliance, or simply a paid tier. Ask for scope, method, tested version, date, exclusions, and remediation path. If that detail is not available, state the limitation rather than interpreting the badge as a guarantee.

A compact review worksheet

Before the review meeting, fill in the following sentence and attach the passport: “We will install [publisher/package/version] from [source] to [task] using [capabilities and credentia...