You have a DOI, the publisher's page wants a login, and you only need to read the methods section. Very often there is a legal free copy somewhere: the author's university repository, a preprint, PubMed Central. Finding it by hand means trying four or five sites. Finding it in code takes a handful of public APIs, called in a sensible order.

The order below is the one used by the View button in the Paper workspace of the osFoundry desktop app, whose backend tries each source in turn until one returns a real PDF (the paper writing and research workspace page shows the screen around it). The examples use two real DOIs on the same question, whether a phone lying on the desk hurts concentration:

  • 10.1037/tmb0000123, a 2024 meta-analysis (Hartanto et al.)
  • 10.1086/691462, the much-cited 2017 "Brain Drain" study (Ward et al.)

View on a search result opens the meta-analysis text inside the app

Ask OpenAlex for every location, not just the open-access flag

OpenAlex lists each place a paper is hosted, and for each one whether there's a direct PDF:

curl -s "https://api.openalex.org/works/doi:10.1037/tmb0000123?select=open_access,best_oa_location,locations"

For the meta-analysis that returns three locations: the publisher's PDF link, a PDF in James Cook University's repository, and a landing page at Singapore Management University's library.

For the Brain Drain paper the same request shows "is_oa": true and "oa_status": "green", yet every pdf_url is null. The only open copy is a repository landing page. So is_oa tells you a free version exists somewhere, not that you can download one. Read pdf_url on each location.

Without an API key, OpenAlex requests share a small free daily budget with everyone on your IP address, and the response's meta.notice says so. A free key (api_key= on the query) gives you your own budget, and the app reads one from OPENALEX_API_KEY if you set it.

Check the first five bytes

A pdf_url is still only a promise. Publishers return cookie walls, bot checks and login pages with a 200 status and an HTML body. The app's Rust code accepts a download only if the body starts with the PDF magic number:

if !fetched.bytes.starts_with(b"%PDF-") { self.tried.push(format!("{source}: link was a web page, not a PDF")); return None; }

The same idea in Python, using only the standard library:

import json import urllib.error import urllib.request UA = {"User-Agent": "paper-finder/1.0"} def get(url): with urllib.request.urlopen(urllib.request.Request(url, headers=UA), timeout=30) as r: return r.read() def pdf_links(doi): try: work = json.loads(get(f"https://api.openalex.org/works/doi:{doi}" "?select=best_oa_location,locations")) except urllib.error.HTTPError as e: if e.code == 404: return [] raise links = [] for loc in [work.get("best_oa_location")] + work.get("locations", []): url = (loc or {}).get("pdf_url") if url and url not in links: links.append(url) return links def first_real_pdf(doi): tried = [] for url in pdf_links(doi)[:4]: try: body = get(url) except (urllib.error.URLError, TimeoutError) as e: tried.append(f"{url}: {e}") continue if body[:5] == b"%PDF-": return url, body tried.append(f"{url}: a web page, not a PDF") raise LookupError("no free PDF; tried: " + "; ".join(tried or ["no pdf_url at all"]))

Run on the meta-analysis, the publisher link refused the script and the repository copy came back as a 1.6 MB PDF. Run on Brain Drain, it fails straight away with "no pdf_url at all". Keeping the list of what was tried and why it failed is worth the extra lines; "no full text" on its own sends you back to doing it by hand.

The cap of four links per lookup and the de-duplication of URLs are also from the app. Some papers have a dozen locations that all point at the same file.

The rest of the chain

The OpenAlex lookup is only one step. The app's full order is:

  1. The PDF link that came with the search result, if there was one.
  2. arXiv. A DOI of the form 10.48550/arXiv.<id> is an arXiv paper, and https://arxiv.org/pdf/<id> is always free.
  3. Open copies of the DOI, from Unpaywall or OpenAlex. Unpaywall (https://api.unpaywall.org/v2/<doi>?email=<your email>) returns best_oa_location.url_for_pdf and a list of oa_locations, and refuses requests without a real email address. The two services come from the same nonprofit, OurResearch, so the app asks Unpaywall when an UNPAYWALL_EMAIL is set and reads OpenAlex's locations otherwise, not both.
  4. CORE. Its v3 API (/v3/search/works?q=doi:"<doi>", with a bearer key) aggregates repository copies and often returns the extracted text directly in fullText. The app takes that text if it's longer than 500 characters, and otherwise tries the downloadUrl.
  5. PubMed Central. With a PMCID, Europe PMC serves the whole article as JATS XML at https://www.ebi.ac.uk/europepmc/webservices/rest/<PMCID>/fullTextXML. It isn't a PDF, but it has the sections, figures and references in a structure that converts cleanly to HTML.

Abstracts come as an inverted index

One more OpenAlex detail that trips people up: there is no abstract field. You get abstract_inverted_index, a map from each word to the positions where it appears, and you rebuild the text yourself:

words = {pos: w for w, positions in idx.items() for pos in positions} abstract = " ".join(words[i] for i in sorted(words))

That is enough for search results and triage. For the Brain Drain paper, whose only open copy is a repository page, the abstract plus the meta-analysis's full text is about as far as the free route goes, and that is usually the point at which to check whether your library has access.