Disclosure: I built the Apify actor used in this post. It is a paid tool (pay per job), and I say below where it doesn't fit.

A spreadsheet of job postings is useful to a lot of people. Job seekers track openings. Recruiters watch the market. Sales teams look for companies hiring for a role their product supports. Researchers compare salaries by city.

Building that spreadsheet by hand is slow, and building it in code has a few traps:

  1. Two boards, two shapes. LinkedIn and Indeed describe the same job with different fields. You need one set of columns before you can sort or filter anything.
  2. The same job shows up twice. A company posts on both boards, or the same posting matches two of your keywords. Without deduplication your counts are wrong.
  3. Salary is messy. Some postings give a yearly range, some an hourly rate, and many give nothing. A blank salary is not a zero salary.
  4. Searches multiply. Three keywords in five cities on two boards is 30 separate searches. That is a lot to run and babysit from a laptop.

This post walks through the task in order: decide on your columns, plan your searches, collect the jobs, then put them in a spreadsheet. At the end we turn it into a daily "new jobs only" alert.

Step 1: decide what a row should look like

Before scraping anything, write down the columns you want. For most uses, this set covers it:

Column Why you want it
title, company, location The basics for reading and filtering
jobBoard Which board the row came from (indeed, linkedin, ...)
salaryMin, salaryMax, salaryCurrency, salaryInterval Numbers you can sort, plus the unit (yearly, hourly)
datePosted Fresh postings matter more than old ones
jobUrl, directApplyUrl The listing, and the employer's own application page when known
searchTerm, searchLocation Which of your searches found the row

The last two columns are easy to forget. When you search several keywords in several cities, you want to know which search produced each job. That lets you count postings per city or per keyword later.

Step 2: plan your searches

The number of searches is keywords x locations x boards. Two keywords in three locations on two boards is 12 searches.

Some rules of thumb:

  • Start small. Ask for 20 to 50 results per search first and check that the jobs are relevant.
  • LinkedIn is slower than Indeed. Its public guest search returns pages of about 10 jobs, with pauses in between. Keep it at 100 results per search or fewer, and add more locations instead.
  • Filter titles, not just keywords. A search for "data engineer" also returns interns and senior managers. Excluding words like intern in the title removes most of that noise.
  • Limit the age. For a market snapshot, postings from the last few days are usually enough.

Step 3: collect the jobs with Python

I use Job Scraper (LinkedIn Jobs, Indeed and multi-city alerts), an Apify actor I built for this. It searches LinkedIn and Indeed without a login, cookies or job-board API key. Every row has the same fields whatever board it came from, and shared listings are merged so you don't pay twice.

You need a free Apify account and your API token (Apify Console, then Settings, then API & Integrations). Then pip install "apify-client>=3" and run this:

import csv from apify_client import ApifyClient client = ApifyClient("YOUR_APIFY_TOKEN") run = client.actor("groupoject/multi-job-board-scraper").call(run_input={ "searchTerms": ["data engineer", "analytics engineer"], "locations": ["New York, NY", "Austin, TX", "Remote"], "jobBoards": ["indeed", "linkedin"], "hoursOld": 72, # posted in the last 3 days "excludeTitleKeywords": ["intern", "principal"], "maxResultsPerSite": 25, # per board, per term, per location "maxItems": 300, # hard cap for the whole run "countryIndeed": "USA", }) COLUMNS = [ "title", "company", "location", "jobBoard", "isRemote", "salaryMin", "salaryMax", "salaryCurrency", "salaryInterval", "datePosted", "jobUrl", "directApplyUrl", "searchTerm", "searchLocation", ] count = 0 with open("jobs.csv", "w", newline="", encoding="utf-8") as f: writer = csv.DictWriter(f, fieldnames=COLUMNS, extrasaction="ignore") writer.writeheader() for job in client.dataset(run.default_dataset_id).iterate_items(): writer.writerow(job) count += 1 print(f"Saved {count} jobs to jobs.csv") 

call() starts the run, waits for it to finish and returns the run details. The jobs are in the run's dataset, which you read with iterate_items(). Open jobs.csv in Excel, Numbers or Google Sheets and you have your spreadsheet.

A few input fields worth knowing (these are the real names from the actor's input schema):

  • searchTerm / location take a single keyword and place. searchTerms / locations take lists, with up to 20 locations per run.
  • includeTitleKeywords keeps only titles that contain one of your phrases. excludeTitleKeywords wins when both match.
  • excludeCompanies drops exact company names, for example your own employer.
  • salaryOnly keeps only jobs with a numeric salary. It does not estimate missing ones.
  • normalizeAnnualSalary converts hourly, daily, weekly and monthly ranges to annual estimates.
  • remoteOnly, jobType, distance and easyApply narrow the search further.
  • linkedinFetchDescription fetches full LinkedIn descriptions and direct apply URLs. It is slower, so leave it off unless you need the text.

Filters run after scraping, so you can get fewer rows than you asked for. That is expected.

What a row looks like

Here is one row (long values shortened):

{ "title": "Senior Software Engineer", "company": "Acme", "location": "New York, NY, US", "jobBoard": "indeed", "jobType": "fulltime", "isRemote": false, "salary": "$120,000 - $150,000 / year", "salaryMin": 120000, "salaryMax": 150000, "salaryCurrency": "USD", "salaryInterval": "yearly", "datePosted": "2026-10-20", "jobAgeDays": 3, "jobUrl": "https://www.indeed.com/viewjob?jk=...", "directApplyUrl": "https://careers.acme.com/jobs/123", "companyIndustry": "Software", "description": "...", "hiringSignalScore": 82, "searchTerm": "software engineer", "searchLocation": "New York, NY" } 

Rows can also carry companyWebsite, companySize, companyRating, skills, jobLevel and other fields. They are filled only when the board publishes them, and company details come mostly from Indeed.

Step 4: from rows to answers with pandas

A CSV is fine for reading. For questions like "what do these jobs pay in each city?", load the rows into pandas (pip install pandas openpyxl):

import pandas as pd jobs = pd.DataFrame(list(client.dataset(run.default_dataset_id).iterate_items())) yearly = jobs[jobs["salaryInterval"] == "yearly"] print(yearly.groupby("searchLocation")[["salaryMin", "salaryMax"]].median()) print(jobs.groupby(["searchLocation", "jobBoard"]).size()) jobs.to_excel("jobs.xlsx", index=False) 

Two habits keep the numbers honest. Compare salaries only within one salaryInterval. And count rows without a salary as "unknown", never as zero.

If you prefer no code for this part, the Apify Console exports any dataset as CSV, Excel or JSON. There is also a Google Sheets integration.

Step 5: a daily "new jobs only" alert

A one-off spreadsheet goes stale in days. For a running feed, turn on onlyNewJobs and give the monitor a stateKey:

run = client.actor("groupoject/multi-job-board-scraper").call(run_input={ "searchTerms": ["data engineer"], "locations": ["New York, NY", "Remote"], "jobBoards": ["indeed", "linkedin"], "hoursOld": 48, "maxResultsPerSite": 30, "onlyNewJobs": True, "stateKey": "nyc-data-jobs", }) 

The first run delivers every match. Later runs with the same stateKey deliver only jobs that were not delivered before. Jobs removed by your filters or by the output cap are not marked as seen, so they can still arrive later.

To run it every morning without a server:

  1. Save the input as a task in Apify Console.
  2. Schedules, then Create: run the task once a day.
  3. Send new rows to Slack, email or Google Sheets with an Apify integration, a webhook, Make, Zapier or n8n. The actor itself does not send notifications.

Use a different stateKey for each alert, and don't run two monitors with the same key at the same time.

Did a board fail, or were there just no new jobs?

A run with zero rows usually means "nothing new". To be sure, read the SEARCH-OUTCOMES record. It has one entry per board, keyword and location, with returned and delivered counts and the time each search took:

store = client.key_value_store(run.default_key_value_store_id) record = store.get_record("SEARCH-OUTCOMES") print(record["value"] if record else "no outcomes record") 

Each search reports ok, empty or failed. If one board fails and another delivers jobs, the run still succeeds and SUMMARY.partialResults is true. If every search fails, the run fails.

What it costs

$2.00 per 1,000 jobs on the free plan ($0.002 per job), down to $1.85 per 1,000 on higher Apify plans. 100 jobs cost $0.20. There is also a start charge of $0.00005 per GB of memory per run. You pay for jobs saved to the dataset, not for searches that return nothing. maxItems (default 100) caps the output, and the actor also stops at your maximum run charge. The Pricing tab on the actor page has the current rates.

Honest limits

  • Indeed and LinkedIn are the tested boards. Google Jobs, Glassdoor, ZipRecruiter, Bayt and Naukri are beta. They are often challenged by anti-bot systems and may return nothing. Add them one at a time and check SEARCH-OUTCOMES.
  • Fields depend on the board. Missing salary, job type or remote flag means the board didn't publish it.
  • LinkedIn is slow. Big LinkedIn searches can hit the run timeout. Split large plans into several saved tasks. Jobs saved before a timeout stay in the dataset.
  • hiringSignalScore is a heuristic. It scores freshness and completeness from 0 to 100. It is not proof that a company is buying anything.
  • Public postings only. The actor collects public job listings without logging in. It does not collect private profiles. You are responsible for how you use the data and for following each board's terms.
  • If you want to run the scraping yourself, you can. The actor is built on the open-source python-jobspy library. With your own machine you then handle proxies, scheduling and the "already seen" state yourself.

Links

If a board returns something odd, open an issue on the actor's Issues tab. Include the board, your input (no tokens) and the run ID. I read them daily.