A lot of useful text is still locked in PDFs: annual reports, tenders, product data sheets, tax forms, research papers. A typical task looks like this: "go through the 40 PDFs on this downloads page and tell me which ones mention X, and on which page".

This post covers:

  1. A short Python script with pypdf that finds every PDF linked from a page, extracts the text and searches it, with page numbers.
  2. The pitfalls: scanned PDFs, phrases broken across lines, links that aren't really PDFs, and very large files.
  3. A hosted bulk extractor that returns text, metadata and search hits as JSON, used from Python and Node.js.

Disclosure: I built the PDF Text Extractor Actor used in part 3. Parts 1 and 2 don't need it.

1. A minimal version (Python + pypdf)

pypdf is pure Python and needs no system libraries. Everything else (finding links, downloading files) uses the standard library. The demo page is the IRS page for Form W-9, which links to three PDFs.

"""Find every PDF linked from a page, extract the text and search it (with page numbers). Usage: pip install pypdf && python pdf_pypdf.py https://www.irs.gov/forms-pubs/about-form-w-9 "backup withholding" """ import io import re import sys import urllib.request from html.parser import HTMLParser from urllib.parse import urljoin from pypdf import PdfReader UA = {"User-Agent": "pdf-search-demo/1.0 (+https://example.com/contact)"} MAX_BYTES = 25 * 1024 * 1024 class Links(HTMLParser): def __init__(self, base): super().__init__() self.base, self.links = base, [] def handle_starttag(self, tag, attrs): href = dict(attrs).get("href") if tag == "a" else None if href and re.search(r"\.pdf($|[?#])", href, re.I): url = urljoin(self.base, href).split("#")[0] if url not in self.links: self.links.append(url) def get(url): with urllib.request.urlopen(urllib.request.Request(url, headers=UA), timeout=30) as r: return r.read(MAX_BYTES + 1) def search_pdf(url, term): data = get(url) if len(data) > MAX_BYTES or not data.startswith(b"%PDF"): # check magic bytes, not the URL  return None reader = PdfReader(io.BytesIO(data)) hits, words = [], 0 for no, page in enumerate(reader.pages, 1): text = re.sub(r"-\n(\w)", r"\1", page.extract_text() or "") # undo hyphenation at line ends  text = re.sub(r"\s+", " ", text) words += len(text.split()) for m in re.finditer(re.escape(term), text, re.I): hits.append((no, text[max(0, m.start() - 50):m.end() + 50])) return len(reader.pages), words, hits page, term = sys.argv[1], sys.argv[2] parser = Links(page) parser.feed(get(page).decode("utf-8", "replace")) for url in parser.links[:3]: # small demo  result = search_pdf(url, term) if result is None: print(f"skip (not a PDF or too big): {url}") continue pages, words, hits = result print(f"{url.rsplit('/', 1)[-1]}: {pages} pages, {words} words, {len(hits)} hits") for no, snippet in hits[:2]: print(f" p.{no}: ...{snippet}...")

Output (03.11.2026):

$ python pdf_pypdf.py https://www.irs.gov/forms-pubs/about-form-w-9 "backup withholding" fw9.pdf: 6 pages, 6278 words, 33 hits p.1: ...ided must match the name given on line 1 to avoid backup withholding. For individuals, this is generally your social s... p.1: ...r to be issued to me); and 2. I am not subject to backup withholding because (a) I am exempt from backup withholding, ... iw9.pdf: 7 pages, 7114 words, 43 hits p.1: ...S.gov/irb/2022-52_IRB#REV-PROC-2022-43. Reminders Backup withholding rate. The backup withholding rate is 24% for repo... p.1: ...C-2022-43. Reminders Backup withholding rate. The backup withholding rate is 24% for reportable payments. Foreign Acco... p1281.pdf: 102 pages, 58613 words, 179 hits p.1: ...BACKUP WITHHOLDING FOR MISSING AND INCORRECT NAME/TIN(S) Publication... p.2: ... ii Backup Withholding For Missing and Incorrect Name/Tin(S) Table of Co...

All three PDFs, 115 pages in total, took about 2.5 seconds on a small VM. For a one-off job on a single page, this is all you need.

2. Pitfalls

  • Phrases get split by line breaks. extract_text() returns the text with the PDF's own line breaks, so "backup\nwithholding" doesn't match "backup withholding". The script collapses all whitespace before searching. Without that step, my own first version of the hosted Actor counted 29 hits in fw9.pdf instead of 33. Hyphenated words at line ends ("with-\nholding") have the same problem, which is why the script also joins those.
  • Scanned PDFs have no text. A scan is just images, so extract_text() returns empty strings. If almost every page comes back empty, flag the file as "probably scanned" instead of reporting "0 hits". Only OCR (for example Tesseract via ocrmypdf) helps there, and that's a much heavier job.
  • A .pdf link isn't always a PDF. Some links return an HTML login page, a cookie wall or a 404 page with status 200, and some servers send PDFs as application/octet-stream. Check the first bytes for %PDF instead of trusting the URL or the Content-Type.
  • Reading order isn't guaranteed. Multi-column layouts, tables and form fields can come out in an odd order. Search hits are reliable, but don't expect clean paragraphs from complex layouts.
  • Large and encrypted files. Cap the download size (the script reads at most 25 MB) and the number of pages if you only need the summary. PDFs with an owner password but no user password usually open fine. PDFs that need a password to open can't be read, and that should be reported as a result, not as a crash.
  • Be polite. Respect robots.txt when you crawl pages for links, download a few files in parallel at most, and don't re-download unchanged files. Checking the Last-Modified or ETag header first saves time.

3. Hosted option: PDF Text Extractor on Apify

The PDF Text Extractor takes PDF URLs or web pages that link to PDFs and returns one record per PDF (or one per page). Each record includes:

  • The full text (optionally truncated), word count and page count
  • Metadata: title, author, creator, creation and modification dates
  • Search hits with page numbers and a snippet; prefix a term with re: for a regular expression
  • likelyScanned for files without a text layer, and a readable error for failed downloads, password-protected files and files that aren't PDFs

It respects robots.txt for pages it scans and checks the magic bytes of every download.

Set APIFY_TOKEN from a free Apify account (Console → Settings → API & Integrations).

Python (pip install apify-client) 

"""Find every PDF linked on a page, extract the text and show where your search terms appear. Actor: https://apify.com/paperlinenstudio/pdf-text-extractor Usage: APIFY_TOKEN=... python pdf_search.py https://www.irs.gov/forms-pubs/about-form-w-9 "backup withholding" """ import os import sys from apify_client import ApifyClient page = sys.argv[1] if len(sys.argv) > 1 else "https://www.irs.gov/forms-pubs/about-form-w-9" terms = sys.argv[2:] or ["backup withholding"] client = ApifyClient(os.environ["APIFY_TOKEN"]) run = client.actor("paperlinenstudio/pdf-text-extractor").call( run_input={ "urls": [page], # PDF URLs or pages that link to PDFs  "findPdfLinks": True, "maxPdfsPerPage": 3, # small demo  "searchTerms": terms, # prefix with re: for a regex  "maxTextLength": 2000, # we only need the hits here  } ) for doc in client.dataset(run.default_dataset_id).iterate_items(): if doc.get("error"): print(f"FAILED {doc['url']}: {doc['error']}") continue print(f"{doc['fileName']}: {doc['pageCount']} pages, {doc['wordCount']} words, hits {doc['matchCounts']}") for m in doc["matches"][:3]: print(f" p.{m['page']}: ...{m['snippet']}...")

Output (03.11.2026):

$ python pdf_search.py fw9.pdf: 6 pages, 6278 words, hits {'backup withholding': 33} p.1: ...appropriate box. The TIN provided must match the name given on line 1 to avoid backup withholding. For individuals, this is generally your social security number (SSN). However,... p.1: ...r (or I am waiting for a number to be issued to me); and 2. I am not subject to backup withholding because (a) I am exempt from backup withholding, or (b) I have not been notifie... p.1: ...me); and 2. I am not subject to backup withholding because (a) I am exempt from backup withholding, or (b) I have not been notified by the Internal Revenue Service (IRS) that I a... iw9.pdf: 7 pages, 7114 words, hits {'backup withholding': 43} p.1: ...Proc. 2022-43, available at IRS.gov/irb/2022-52_IRB#REV-PROC-2022-43. Reminders Backup withholding rate. The backup withholding rate is 24% for reportable payments. Foreign Accou... p.1: ...RS.gov/irb/2022-52_IRB#REV-PROC-2022-43. Reminders Backup withholding rate. The backup withholding rate is 24% for reportable payments. Foreign Account Tax Compliance Act (FATCA)... p.1: ...is 24% for reportable payments. Foreign Account Tax Compliance Act (FATCA) and backup withholding exemptions. FATCA requires a participating foreign financial institution (FFI)... p1281.pdf: 102 pages, 58613 words, hits {'backup withholding': 179} p.1: ...BACKUP WITHHOLDING FOR MISSING AND INCORRECT NAME/TIN(S) Publication 1281 (Rev. 12-2023) Catalog N... p.2: ...ii Backup Withholding For Missing and Incorrect Name/Tin(S) Table of Contents Part 1 Introduction....... p.2: ................................................................................ 1 Backup Withholding Rate .............................................................................

The counts match part 1. The value of the hosted version is that the same call works for 500 PDFs spread over 30 pages, runs in parallel, and gives you failed and scanned files as data instead of exceptions.

Node.js (npm i apify-client) 

// Find every PDF linked on a page, extract the text and show where your search terms appear. // Usage: APIFY_TOKEN=... node pdf_search.mjs https://www.irs.gov/forms-pubs/about-form-w-9 "backup withholding" import { ApifyClient } from 'apify-client'; const page = process.argv[2] ?? 'https://www.irs.gov/forms-pubs/about-form-w-9'; const terms = process.argv.slice(3).length ? process.argv.slice(3) : ['backup withholding']; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client.actor('paperlinenstudio/pdf-text-extractor').call({ urls: [page], // PDF URLs or pages that link to PDFs findPdfLinks: true, maxPdfsPerPage: 3, searchTerms: terms, // prefix with re: for a regex maxTextLength: 2000, }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); for (const doc of items) { if (doc.error) { console.log(`FAILED ${doc.url}: ${doc.error}`); continue; } console.log(`${doc.fileName}: ${doc.pageCount} pages, ${doc.wordCount} words, hits ${JSON.stringify(doc.matchCounts)}`); for (const m of doc.matches.slice(0, 3)) console.log(` p.${m.page}: ...${m.snippet}...`); } 
$ node pdf_search.mjs fw9.pdf: 6 pages, 6278 words, hits {"backup withholding":33} iw9.pdf: 7 pages, 7114 words, hits {"backup withholding":43} p1281.pdf: 102 pages, 58613 words, hits {"backup withholding":179}

(Snippet lines omitted here; they're the same as in the Python output.)

A single record (real, from 03.11.2026; text and hit list shortened):

{ "url": "https://www.irs.gov/pub/irs-pdf/fw9.pdf", "sourcePage": null, "finalUrl": "https://www.irs.gov/pub/irs-pdf/fw9.pdf", "fileName": "fw9.pdf", "fileSizeBytes": 140815, "pageCount": 6, "pagesExtracted": 6, "encrypted": false, "title": "Form W-9 (Rev. March 2024)", "author": "SE:W:CAR:MP", "subject": "Request for Taxpayer Identification Number and Certification", "keywords": "Fillable", "creator": "Designer 6.5", "producer": "Designer 6.5", "createdAt": "2024-03-06T08:18:13", "modifiedAt": "2024-03-06T08:18:13", "likelyScanned": false, "emptyPages": 0, "text": "Form W-9\n(Rev. March 2024)\nRequest for Taxpayer\nIdentification Number and Certification\nDepartment of the Treasury …", "textLength": 37692, "wordCount": 6278, "textTruncated": true, "matchedTerms": [ "backup withholding" ], "matchCounts": { "backup withholding": 33 }, "matches": [ { "term": "backup withholding", "page": 1, "snippet": "appropriate box. The TIN provided must match the name given on line 1 to avoid backup withholding. For individuals, this is generally your social security number (SSN). However," }, { "term": "backup withholding", "page": 1, "snippet": "r (or I am waiting for a number to be issued to me); and 2. I am not subject to backup withholding because (a) I am exempt from backup withholding, or (b) I have not been notifie" } ], "pageErrors": 0, "error": null, "durationMs": 276, "extractedAt": "2026-11-03T14:12:28+00:00" }

Inputs you'll probably change 

{ "urls": ["https://example.com/investor-relations/reports"], "searchTerms": ["revenue", "re:net (?:loss|income)"], "onlyMatching": true, "sameSiteOnly": true, "outputMode": "page", "maxPagesPerPdf": 30, "maxTextLength": 0 }

onlyMatching drops PDFs without a hit (those aren't charged). outputMode: "page" returns one item per page, which is handy if you want to feed pages into a search index or an LLM in chunks. maxTextLength: 0 keeps the full text.

Wrap-up

  • One page, a handful of PDFs: pypdf plus the standard library is enough. Collapse whitespace before searching and check for %PDF.
  • Many pages or hundreds of PDFs, with scanned-file detection and errors as data: PDF Text Extractor.

Feedback and feature requests are welcome, either in the comments or on the Actor's Issues tab.