Your Agent Cited Pastebin: 5 Checks Before You Trust HostedWebSearchTool

Someone on the review call said "we're grounded now — we added web search." The recording then cited a random blog. Another run answered from memory while web_search still sat in the tool list. A third returned a green path with a Pastebin-shaped URL that nobody blocked.

That is not RAG. RAG is retrieval over a corpus you control. HostedWebSearchTool in Microsoft.Extensions.AI 10.11 is a marker. Capable backends (OpenAI Responses web_search, Agent Framework hosted search, peers) may browse. The class does not search locally. Microsoft Learn says so plainly: the tool does not itself implement web searches.

This post is not a full wiring guide. It is five checks I run on the response after the marker returns, with the shape of each fix. Snippets are truncated on purpose.

Tech Skill Builder is $5/month for a very limited time only (normally $10). Grab it now before it goes back to $10. The complete, working, tested project behind this post (hosted-web-search-citations-dotnet) is inside, with full source and an in-depth PDF guide. Join Tech Skill Builder

Check 1: Did a search actually happen?

"Tool registered" ≠ "answer grounded." If the assistant writes fluent prose with zero WebSearchToolCallContent, zero result URIs, and zero CitationAnnotations, that text is parametric memory wearing a tool costume.

var hadSearch = extracted.Queries.Count > 0 || extracted.Citations.Count > 0 || response.Messages .SelectMany(m => m.Contents) .Any(c => c is WebSearchToolCallContent or WebSearchToolResultContent); if (!hadSearch) { // Refuse. Do not return the model text as "web-grounded." // (full refusal + audit wiring in the complete project) } 

Prefer the Responses API path (GetResponsesClient().AsIChatClient() or equivalent). Chat Completions has a documented footgun where HostedWebSearchTool plus a bad tool_choice returns HTTP 400. The maintainers point at Responses as the workaround.

Check 2: Who is allowed to appear as a source?

Provider allowed_domains / blocked_domains are a courtesy. Your process still owns the verdict.

public bool IsAllowed(string domain) { if (IsDenied(domain)) return false; if (_allow.Count == 0) return !_failClosed; var d = Normalize(domain); return _allow.Any(allowed => d == allowed || d.EndsWith("." + allowed, StringComparison.Ordinal)); } // Normalize: strip scheme, www., path, trailing dots; compare lowercase. // Subdomain suffix match; refuse lookalikes like learn.microsoft.com.evil.test. // Overlap between allow and deny should throw at construction. // (full DomainPolicy in the complete project) 

Default sample allow: Learn, docs.microsoft.com, nuget.org, dotnet.microsoft.com, DevBlogs. Deny: Reddit, Quora, Medium, Blogspot, WordPress, Pastebin. Empty allow + fail-closed rejects everything. That sounds harsh until you ship an agent that cites Pastebin because someone forgot to load config.

A denied host is not a soft log line. It must not render as a clickable source chip. "One source removed by policy" is enough transparency without handing the user the hostile URL.

Check 3: Did you upgrade duplicates, or keep the worse record?

MEAI surfaces citations in more than one shape:

• WebSearchToolCallContent — CallId + Queries
• WebSearchToolResultContent — CallId + Outputs (often UriContent)
• CitationAnnotation on TextContent — Url, Title, Snippet, optional span

If you ingest result URIs first and annotations second, a naïve HashSet of absolute URIs keeps the bare URI and drops the annotation that carried the date and title. Upgrade the richer record. That single merge rule is why a conflicting-dates demo can prefer August over June later.

Dates need the same honesty. If an ISO yyyy-MM-dd appears in the title or snippet, mark it approximate. Otherwise UnverifiedDate. Soft policies can still use it. Strict policies can treat it as stale. Inventing a crawl time from thin air is how you lie to your own freshness gate.

Check 4: Is the snippet data — or an instruction?

Allowlisted hosts can still serve hostile content (compromised mirror, injected docs page). Treat every snippet as untrusted until a fence says otherwise.

public static string FenceSnippet(string? snippet, string domain) { // Wrap in a clear "DATA ONLY" block tagged with the domain. // Detect: ignore-previous / you-are-now family, role delimiters, // hidden bidi / zero-width chars. Case-insensitive. // Cap length; strip nulls and hidden chars. // (full PromptFence in the complete project) } // If title or snippet looks like injection → InjectedInstruction // even when the host is on the allow list. 

Showing the blocked blog "so the user understands" reopens the surface you just closed. Partial mode keeps survivors and drops junk. Render only survivors.

Check 5: What mode did the caller get?

Collapse policy into something monitors can alert on:

Usable means Accepted or UnverifiedDate. Stale, denied, not-allowlisted, malformed, and injection all count as blocked.

var answer = await new GroundedWebSearchClient( chatClient, DomainPolicy.MicrosoftDocsDefault(), new FreshnessOptions { MaxAge = TimeSpan.FromDays(365) }) .AskAsync(question); // answer.Mode is what ops watches. // HTTP stays 200 on refuse; turning refusals into 500s trains clients to retry worse. // Audit: queries, domains, blocked counts — not page bodies. // Offline fake throws if the marker is missing (CI catch for "temporary" tool removal). // (full client + 9 offline scenes + metrics in the complete project) 

What this is not

It is not hybrid BM25 + vectors. It is not permission-aware RAG. It is not MCP brokering. Hosted web search solves "public facts with URLs from a provider," and only if you treat those URLs as claims to verify after the model speaks. Keep web citations labeled differently from RAG chunk ids in telemetry.

Ship checklist (short)

1. Responses client (or a provider that maps the marker), not Completions alone.
2. Provider filters and in-process allow/deny.
3. Injection fence on titles and snippets, allowlist or not.
4. Modes exposed; refusals monitored without becoming 500s.
5. Audit without page bodies; offline test that fails if the marker disappears.

None of that is glamorous. The glamorous part is the demo GIF with three blue links. The durable part is refusing the fourth when it points at a paste site.

When two allowlisted domains contradict each other on a dated fact, how do you pick a winner without trusting whichever snippet the model preferred?

The full answer — search budget, conflict ladder, and the offline demos — is in the complete project.

Get the complete project

ZIP + in-depth PDF live in Tech Skill Builder: full .NET 10 solution (hosted-web-search-citations-dotnet), nine offline scenes, API, multi-hop budget, and conflict resolution this post deliberately leaves closed.

Tech Skill Builder is only $5/month for a very limited time (normally $10). Grab it now before it goes back to $10.

Join Tech Skill Builder