PR teams, analysts and anyone watching a brand ask Google News the same things every day: what was written about us, by whom, and what is new since yesterday. Two things get in the way when you try to collect it: a search shows at most 100 articles, and every link points at a Google redirect instead of the publisher's page.
This post reads Google News into Python by keyword, topic or site, gets the publisher's own article links, goes past 100 results over a longer window, pulls full text for an AI summary, and sets up a daily feed of only the new articles.
Disclosure: I'm Aurelix, and I built Google News Scraper, the tool used here. It's a paid tool (prices below).
What each row holds
One row per article:
-
status(ok, or why nothing came back:no_results,failed,skipped,unsupported_edition,invalid_input),error. -
title,publisher,publisher_url,url(the decoded publisher link),url_resolved,google_news_url,published_at(ISO 8601 UTC),snippet. -
queryortopic,language,country,edition,article_id. - With full text on:
full_text(Markdown),word_count,full_text_status.
Articles for a few keywords
pip install apify-client import csv import os from apify_client import ApifyClient client = ApifyClient(os.environ["APIFY_TOKEN"]) run = client.actor("aurelix/google-news-scraper").call( run_input={ "queries": ["electric vehicles", '"interest rates" -crypto'], "timeWindow": "7d", "maxArticlesPerQuery": 100, } ) articles = [a for a in client.dataset(run["defaultDatasetId"]).iterate_items() if a["status"] == "ok"] fields = ["published_at", "query", "publisher", "title", "url"] with open("news.csv", "w", newline="", encoding="utf-8") as f: w = csv.DictWriter(f, fieldnames=fields, extrasaction="ignore") w.writeheader() w.writerows(sorted(articles, key=lambda a: a["published_at"] or "", reverse=True)) Google News search operators work in queries (quotes, -word, site:). timeWindow takes the last hour (1h), day (1d), week (7d), month (30d), any, or custom with dateFrom and dateTo for an exact date range. language and country pick the edition, e.g. fr and FR.
Who covered it most
from collections import Counter for publisher, n in Counter(a["publisher"] for a in articles).most_common(10): print(f"{n:>4} {publisher}") url is the publisher's own article link, decoded from Google's redirect at no extra cost; the Google News link stays in google_news_url. If Google does not answer a look-up, url_resolved is false and url keeps the Google News link.
More than 100 articles
A single Google News search answers with at most 100 articles. Set maxArticlesPerQuery above 100 with a window of a day or more, and the window is read in date slices down to single days and deduplicated until your number is reached:
run_input = {"queries": ["interest rates"], "timeWindow": "30d", "maxArticlesPerQuery": 1000} The run's status message counts the days that returned 85 or more articles; those days may hold more than Google shows, so narrow the query for them.
Only some publishers, with full text for an AI summary
run = client.actor("aurelix/google-news-scraper").call( run_input={"queries": ["electric vehicles"], "sites": ["electrek.co", "insideevs.com"], "timeWindow": "7d", "maxArticlesPerQuery": 50, "includeFullText": True} ) for a in client.dataset(run["defaultDatasetId"]).iterate_items(): if a.get("full_text"): print(a["title"], a["word_count"], "words") elif a["status"] == "ok": print(a["title"], "-", a["full_text_status"]) Full text is read from the publisher's site only where its robots.txt allows it, as Markdown with a word count. Paywalled pages, disallowed pages and pages without readable text are skipped, say why in full_text_status, and are not charged.
Only new articles, every morning
run_input = { "queries": ['"Acme Corp"', '"Acme Rocket"'], "timeWindow": "1d", "maxArticlesPerQuery": 100, "incremental": True, "stateKey": "acme-daily", "slackWebhookUrl": "https://hooks.slack.com/services/...", } The first run returns everything that matches and remembers it. Each later run returns only articles earlier runs did not return; a search with nothing new is free for the first 10 in a run, then $0.0001 each. Keep stateKey set so the memory stays the same while you add or remove queries. Save the input as a task in the Apify Console and schedule it daily: Slack gets a short digest, and nothing is sent when nothing is new.
Why this tool
- More than 100 results per search: long windows are split into date ranges and deduplicated.
- The publisher's real article link on every row, included in the price.
- Any edition, any window from the last hour to a month, or an exact date range.
- Optional full text as Markdown; skipped pages are free.
- An only-new mode with a Slack or webhook digest.
Limits
- A single day with more than 100 articles for a query cannot be split further; narrow the query for those days.
- Last hour and
anyare single searches, up to 100 articles each. - Search results carry no summary:
snippetis filled from the article itself when full text is on, otherwise it is mostly empty. - Topic sections list their current articles (about 30 to 80).
- Full text skips paywalled pages, sites whose robots.txt disallows it and pages built only with JavaScript.
- If Google refuses the run, it stops with a BLOCKED status rather than retrying without end.
Cost
$1.00 per 1,000 articles on Apify's free plan, from $0.70 per 1,000 on paid plans, plus a $0.005 start fee per run; full text is $1.00 per 1,000 articles that come back with text. The current prices are on the Store page; Apify's free plan includes $5 of credit a month, about 4,700 articles.
Use it from an AI assistant
The tool also works through Apify's hosted MCP server, so Claude, ChatGPT or Cursor can run it from a request like "get every Google News article mentioning our brand in the last 24 hours with the real article links". Server URL: https://mcp.apify.com/?tools=aurelix/google-news-scraper.