I've been exploring ideas for a small project where I could use Anthropic's Claude API to solve a practical problem rather than build another generic AI chatbot for a client.
One idea that caught my attention is something I'm calling PolicyLens.
The problem I'm trying to solve
Government policies, public schemes, regulatory notifications and official circulars often contain information that people genuinely need. But finding that information and understanding what it actually means are two different things.
A document might be 30–50 pages long, filled with formal language, eligibility conditions, exceptions, application procedures and deadlines.
Someone might only want to know:
- Am I eligible?
- What documents do I need?
- What benefits are available?
- Is there a deadline?
- How do I apply?
Instead of going through the entire document or relying on someone else to interpret it, what if people could simply upload the document and ask questions?
That's the basic idea behind PolicyLens.
How I'm thinking of building it
I'm considering a fairly simple web application to start with.
The current stack I'm exploring is:
- React + Vite for the frontend
- Python + FastAPI for the backend
- PostgreSQL with pgvector for document storage and retrieval
- Anthropic Messages API for document understanding and question answering
- Claude PDF support for processing documents containing text, tables, charts and visual elements
- Voyage AI embeddings for semantic search
- PyMuPDF or pdfplumber for PDF extraction
- OCR for scanned documents, where necessary
One thing I'm particularly interested in is how Claude's native PDF understanding and document citation capabilities could fit into this architecture.
Anthropic doesn't currently provide its own embedding model, so I'd need a separate embedding provider such as Voyage AI if I decide to implement a custom RAG pipeline.
I'm still trying to figure out whether I need to build the entire retrieval pipeline myself or whether Claude's long context window and document processing capabilities could simplify some of that work.
Getting information out of the PDF
The first challenge is document extraction.
Claude supports PDF processing through the Messages API. It can analyze both extracted text and visual content, including charts, tables and diagrams.
My initial thought is to retain a preprocessing layer rather than send every document directly to the model without any preparation.
I'd probably start with PyMuPDF and check whether each page contains usable text. Pages with no extractable text could be routed through OCR.
For documents that contain complex layouts, tables or scanned pages, Claude's visual PDF understanding could provide another layer of analysis.
I'd also like to retain page numbers, section headings and document metadata during extraction. Otherwise, even if the model gives a correct answer, showing the user where that information came from could become difficult.
One concern is that Claude's native document citations for PDFs depend on extractable text. Scanned documents may require OCR and a separate source-mapping mechanism if I want reliable references.
What happens after extraction?
Once the document is available, I'm thinking of using Claude's Messages API to identify important information.
For example:
- Policy or scheme name
- Issuing authority
- Notification number
- Effective date
- Eligibility conditions
- Benefits and financial limits
- Required documents
- Application procedure
- Deadlines
- Exceptions and exclusions
I'm considering Claude's Structured Outputs capability here, using JSON Schema to ensure that the extracted information follows a predefined structure instead of relying on free-form responses.
Something like:
{
"policy_name": "Example Scheme",
"issuing_authority": "Department Name",
"effective_date": null,
"eligibility": [
{
"condition": "Applicant must meet the specified income limit",
"source_page": 4,
"source_section": "Eligibility"
}
],
"required_documents": [],
"benefits": [],
"exceptions": [],
"application_process": []
}
This is only an example of the structure I'm considering.
One thing I'm unsure about is how much of this extraction should happen in a single API call.
Would it be better to process the entire document using Claude's larger context capabilities, or split it into sections and process them independently?
The latter might be easier to validate, but it could also lose context between sections.
Making the document searchable
For question answering, I'm considering splitting the extracted content into smaller chunks and generating embeddings using Voyage AI.
Each chunk would ideally retain its document ID, page number, section heading and version information.
These could be stored in PostgreSQL using pgvector.
When someone asks a question, the system would retrieve the relevant chunks and pass them to Claude through the Messages API to generate an answer.
I'm also wondering whether semantic search alone would be enough.
For example, a question about a particular notification number, income threshold or application date might be better handled through keyword search.
A hybrid approach combining semantic and keyword retrieval could make more sense, but I'd like to understand whether the additional complexity is justified for a project of this size.
Another possibility is to rely more heavily on Claude's long context window and provide larger portions of the original document directly.
I'm not sure where the balance should be between retrieval quality, API costs, response latency and implementation complexity.
The part I'm most interested in: eligibility
This is where I think the project could become more useful than a document summarizer.
Suppose a user uploads a scheme notification and asks whether they qualify.
Claude could extract the eligibility conditions and convert them into structured rules.
For example:
{
"criteria": [
{
"field": "annual_income",
"operator": "<=",
"value": 250000,
"source_page": 4
},
{
"field": "residency",
"operator": "equals",
"value": "Tamil Nadu",
"source_page": 5
}
]
}
The backend could then compare these conditions against information provided by the user.
I would prefer not to let Claude independently decide whether someone is eligible.
It might interpret a condition incorrectly, overlook an exception or make assumptions about information that isn't provided.
A backend rule evaluator seems more appropriate for conditions that can be expressed explicitly.
Claude could help identify conditions that require contextual interpretation, while the backend handles deterministic comparisons.
However, not every policy condition will be this straightforward. Some may depend on multiple conditions, exceptions or interpretations across different sections.
I'm still thinking about how to handle those cases without giving users misleading results.
A few things that could make this complicated
There are some problems I hadn't initially considered.
1. Policy amendments
A newer notification doesn't necessarily replace an entire older policy. It might modify just one eligibility condition or a specific clause.
How would I determine which version of a particular provision is currently applicable?
Simply storing the latest PDF might not be sufficient.
2. Conflicting information
What happens when two sections appear to contain different conditions? Or when an amendment contradicts something in the original document?
I'd rather have the application identify the discrepancy than confidently provide one interpretation.
3. Source accuracy
I don't want answers that merely sound correct.
Claude supports native document citations, which can reference supporting passages and PDF page ranges.
However, I'm still trying to understand how best to combine those citations with custom retrieval, page-level metadata and document versioning.
Would native citations be sufficient for this use case, or would maintaining my own source-mapping mechanism provide better control?
4. Different document formats
A clean, digitally generated PDF is one thing. A scanned circular with tables, stamps and footnotes is another.
Claude's visual PDF processing could help with complex layouts, but scanned documents and OCR accuracy introduce another layer of complexity.
I'm wondering how much preprocessing would be required before the AI layer can reliably understand these documents.
5. Context and API costs
Claude's larger context capabilities are interesting, especially when dealing with lengthy government notifications.
But sending an entire document for every question might not be the most efficient approach.
Would prompt caching, document chunking and retrieval be a better combination for repeated interactions with the same policy?
My initial goal is fairly small: upload a government PDF, extract its important information, and allow users to ask questions with references to the original document.
If that works reliably, I could explore eligibility checks and multilingual support later.
I'm also interested in whether Claude's tool-use capabilities could eventually help connect the extracted policy information to external government portals or application workflows.
I'm curious whether this is a reasonable use case for the Anthropic API, or whether I'm underestimating the complexity of document processing and policy interpretation.
Would appreciate suggestions, alternative architectures, or even things you think are likely to go wrong with this idea.
