The HTTP Request node is the most used node in n8n.
It is also where most scraping workflows die.
Point it at a modern website and you get back a page shell with no data in it, a cookie wall, or a 403. The content you wanted is rendered by JavaScript after the page loads, and the HTTP Request node never runs JavaScript.
If you're:
- automating reports and alerts with n8n,
- tracking websites that have no API,
- or building AI agents that need to read the web,
This tutorial is for you.
I built a complete n8n browser automation workflow with the verified Browser Use node: every day it opens the previous day's Product Hunt leaderboard, reads the top 5 products, appends them to Google Sheets and emails me a summary. Every screenshot, number and cost below comes from my own runs on November 17 and 18, 2026.
TL;DR
- Install: the Browser Use node is verified by n8n, so on n8n Cloud it installs from the nodes panel in one click (package
n8n-nodes-browser-use-cloud)- Setup: one node, "Run a task and wait", plus a JSON schema for the output
- Result: the agent returned Product Hunt's top 5 as clean JSON in 22 to 82 seconds
- Cost: $0.0046 to $0.0124 of agent cost per run, plus browser time and proxy traffic billed separately ($0.031 for the hour of my test runs)
- Gotcha: ask for a day that is already over. "Today" on Product Hunt gave me hidden votes once and an empty list once
Why n8n web scraping breaks on modern sites
There are three usual ways to scrape a website from n8n, and each one breaks somewhere.
HTTP Request + HTML Extract. Fast and free. It only sees the raw HTML the server sends, so anything loaded by JavaScript is invisible, and sites behind bot protection answer with a challenge page.
Self-hosted Puppeteer or Playwright. It renders JavaScript, but now you maintain a headless browser next to n8n. You also write selectors by hand, and they break the day the site changes its CSS classes.
Community scraping nodes. Some are maintained, some haven't been touched in a year. For "browser use n8n", Google's first page still shows an old third-party package that the Browser Use docs explicitly tell you not to install.
All three share the same weakness.
You have to tell the computer how to get the data, step by step.
Describe the task, not the clicks
A browser agent flips that.
You write what you want in one sentence: "Open yesterday's Product Hunt leaderboard and return the top 5 products." The agent opens a real browser in the cloud, reads the page, scrolls or clicks if it needs to, and hands back the answer.
All n8n needs back is clean JSON the next node can use. The Browser Use node does that inside a workflow.
What the Browser Use node does
Browser Use is an open-source browser agent library with a managed cloud version. The cloud version runs the browsers, the stealth layer, the proxies and the agent loop for you, behind one API.
The n8n node wraps that API. After installing it I counted 35 actions in four groups:
- Run actions: create a run, run a task and wait, get a run's status, events or attachments, cancel
- Session actions: keep a browser session open across several tasks
- Browser actions: start a bare cloud browser you control yourself
- Task actions: the older v2 task API, kept for existing workflows
For a browser agent inside a workflow, you only need one of them: Run a task and wait.
👉 If you want to follow along, create a Browser Use Cloud account. It's pay as you go, and signing up gives you a $1 credit, which covered every test in this article many times over.
Step 1: Install the verified node
The node is built by Browser Use and verified by n8n. On n8n Cloud you don't need the Community Nodes settings page:
- Open a workflow and click + to add a node.
- Search for Browser Use. Pick the result with the verified badge.
- Click Install node.
On my instance it installed package version 1.2.1 (marked "Latest").
On a self-hosted n8n, go to Settings → Community Nodes and install n8n-nodes-browser-use-cloud. Don't install n8n-nodes-browser-use: that's the older third-party package, and the Browser Use n8n docs warn against it.
Step 2: Add a Schedule Trigger
Start the workflow with a Schedule Trigger: trigger interval Days, every 1 day, at the hour you want the email. I used 6pm.
The hour doesn't matter much here, because the workflow reads a day that is already closed. It took me two failed runs to get there. More on that in the results section.
Step 3: Configure "Run a task and wait"
Click + after the trigger, search Browser Use, and under Run actions choose Run a task and wait.
The node opens with the right defaults:
- API Version: V4 Runs, Sessions, and Browsers
- Resource: Run
- Operation: Run and Wait
- Wait Timeout: 900
Credential. Click Connect to Browser Use, paste your API key from the Browser Use Cloud dashboard and save. Leave the Base URL field as it is.
Task. Write the job in plain English:
Open this Product Hunt daily leaderboard and return the top 5 products of that day as ranked there: rank, product name, tagline, upvote count, and the Product Hunt URL of each product. Starting URL. Point the agent at yesterday's leaderboard with an n8n expression. Product Hunt runs on Pacific time, so convert the date first:
https://www.producthunt.com/leaderboard/daily/{{ $now.setZone('America/Los_Angeles').minus({ days: 1 }).toFormat('yyyy/M/d') }} Giving the agent its first page saves steps, and a fixed date removes any doubt about which "today" you mean.
Max Cost USD. Under Run Options → Add Option, add Max Cost USD and set a hard cap. I used 0.25. A loop that goes wrong stops there instead of on your invoice.
Step 4: Get structured data back
Turn on Extract Structured Data, choose Custom JSON Schema as the data template, and paste the shape you want:
{ "type": "object", "properties": { "launches": { "type": "array", "items": { "type": "object", "properties": { "rank": { "type": "integer" }, "name": { "type": "string" }, "tagline": { "type": "string" }, "upvotes": { "type": "integer" }, "url": { "type": "string" } }, "required": ["rank", "name", "tagline", "upvotes", "url"] } } }, "required": ["launches"] } Read the yellow notice under the toggle (bottom of the screenshot in Step 3). On API v4 the schema is not enforced by the API: the node appends it to your task as an instruction ("Return the final answer as a single JSON document that conforms to this JSON Schema") and then parses the answer into a parsedResult field on a best-effort basis. The raw text stays in result.
It parsed correctly in every run I made. Still, "best effort" means you check the data before trusting it. Step 7 shows how.
Click Execute step. Here's the output from my full run, straight from n8n:
"parsedResult": { "launches": [ { "rank": 1, "name": "Creatify Max", "tagline": "Your ad teammate, from strategy to creation to launch", "upvotes": 419, "url": "https://www.producthunt.com/products/creatify" }, ... ] } Next to it, the node returns the run's metadata: status, stepCount, durationSeconds, totalCostUsd, the model used, and a sessionId and eventsUrl you can use to replay what the agent did.
Step 5: Split the launches into rows
parsedResult.launches is one array. Google Sheets wants one item per row.
Add a Split Out node and set Fields To Split Out to parsedResult.launches. One item in, five items out.
Step 6: Save to Google Sheets and send the email
Google Sheets. Add Google Sheets → Append row in sheet, sign in with Google, and pick your spreadsheet and tab. Set Mapping Column Mode to Map Automatically. On an empty sheet, the first run writes the field names (rank, name, tagline, upvotes, url) as the header row, and every run after that appends five rows under it.
Email. Add Gmail → Send a message. It reuses the Google sign-in you already did for Sheets. Set:
- To: your address
- Subject:
Product Hunt top 5 · {{ $now.setZone('America/Los_Angeles').minus({ days: 1 }).toFormat('dd LLL yyyy') }} - Email Type: HTML
- Message:
<h3>Product Hunt top 5 · {{ $now.setZone('America/Los_Angeles').minus({ days: 1 }).toFormat('dd LLL yyyy') }}</h3> <ol> {{ $('Split Out').all().map(i => `<li><a href="${i.json.url}">${i.json.name}</a>: ${i.json.tagline} (${i.json.upvotes} upvotes)</li>`).join('') }} </ol> Check that the Subject field is in Expression mode (hover it and pick Expression). In Fixed mode n8n sends the {{ }} text literally, and my first email arrived with the raw expression as its subject.
Under Options, add Append n8n Attribution and turn it off if you don't want n8n's footer in the email.
Then open the Gmail node's Settings and turn on Execute Once. Without it, Gmail receives five items and sends five emails.
Here's what the full run produced. The sheet, with the header row written automatically:
And the email, one message with all five:
Click Publish and the workflow runs on its own every day.
What it actually cost (and the two things that went wrong)
I ran the agent five times on November 17 and 18: twice through the API directly and three times from n8n.
| Run | Started (UTC) | Target | Steps | Duration | Agent cost | Result |
|---|---|---|---|---|---|---|
| API | Nov 17, 19:28 | Homepage, "today" | 6 | 23.2 s | $0.0046 | 5 launches with votes |
| n8n | Nov 18, 09:17 | Homepage, "today" | 9 | 81.8 s | $0.0124 | 5 launches, votes -1 |
| n8n | Nov 18, 10:46 | Homepage, "today" | 7 | 38.8 s | $0.0062 | empty list |
| API | Nov 18, 10:49 | Leaderboard, Nov 17 | 6 | 22.0 s | $0.0056 | 5 products with votes |
| n8n | Nov 18, 10:50 | Leaderboard, Nov 17 | 7 | 27.0 s | $0.0065 | 5 rows in Sheets + email |
The cost column is the totalCostUsd each run reports, and in the run's event log those charges are the model calls. The browser itself and the proxy traffic show up separately, in the Browsers tab of the Browser Use usage dashboard:
$0.031 for the hour of my test runs, and 88% of it was proxy data, not browser time. That matches what I saw in my Browser Use review: on content-heavy pages, proxy traffic is the bill. Even if every daily run cost as much as that whole hour plus the most expensive agent run in the table ($0.031 + $0.0124), a month of daily runs would come to about $1.30.
The middle two runs are the interesting ones.
On the 09:17 run every upvotes value came back as -1, and the agent added a note to the JSON:
"Product Hunt displayed today's upvote counts as —, so -1 indicates unavailable; the ranks follow the homepage listing order."
It was 01:17 in San Francisco. Product Hunt's day had just started and the page wasn't showing vote counts yet. The agent saw a dash, said so, and didn't make up numbers.
An hour and a half later it returned {"launches":[]}. Its reasoning log shows it trying to work out which date "today" meant for a site running on Pacific time. It couldn't settle it, so it returned nothing rather than guess. Honest, but useless for a daily email, and n8n stops the workflow when a node outputs no items.
The fix was in the task, not the tool. I stopped asking for "today" and pointed the agent at yesterday's leaderboard, a page with a fixed date and final vote counts. The next two runs returned the full top 5 in 22 and 27 seconds.
What I took from it:
- Ask for data that already exists. A closed day beats a live one when you need a reliable daily result.
- Validate before you write. A sheet full of
-1is harmless. A sheet full of invented numbers isn't.
Step 7: Guard the output with an IF node
Because v4 parses the schema on a best-effort basis, add an IF node between Browser Use and Split Out. Two conditions:
-
{{ $json.parsedResult.launches.length }}is equal to5 -
{{ $json.parsedResult.launches.every(l => l.upvotes >= 0) }}is true
The true branch goes to Split Out. The false branch sends a short email ("Product Hunt run incomplete, check the session") with the eventsUrl, so you can see what the agent saw.
Prefer code? It's the same API
The node is a thin layer over Browser Use's v4 API. Here's the same run in Python with only the standard library, useful when the agent has to run from your own service instead of n8n:
import json, time, urllib.request API = "https://api.browser-use.com/api/v4" HEADERS = {"X-Browser-Use-API-Key": "YOUR_API_KEY", "Content-Type": "application/json"} SCHEMA = json.load(open("launches_schema.json")) # the JSON schema from Step 4 def call(method, path, body=None): req = urllib.request.Request(API + path, method=method, headers=HEADERS, data=json.dumps(body).encode() if body else None) with urllib.request.urlopen(req, timeout=60) as r: return json.loads(r.read()) run = call("POST", "/runs", { "task": "Go to https://www.producthunt.com/leaderboard/daily/2026/11/17 and return the top 5 " "products of that day's leaderboard: rank, product name, tagline, upvote count, and URL.", "outputSchema": SCHEMA, }) while call("GET", f"/runs/{run['id']}/status")["status"] not in ("completed", "failed", "cancelled"): time.sleep(8) result = call("GET", f"/runs/{run['id']}") print(result["output"], result["totalCostUsd"]) When you call the API directly, outputSchema is a real field, and the structured answer comes back in output. V4 runs don't send webhooks, so you poll the status endpoint, which is what the node does for you while it waits.
Common problems
- You only see "Execute a task" and the old task actions. Those are the v2 actions. Use the Run actions group, or set API Version to V4 on an older node.
- The node times out. Raise Wait Timeout. Agents that log in or paginate need more time than a homepage read.
- The run stops early. Check Max Cost USD. The cap is doing its job; raise it if the task is legitimately long.
-
parsedResultis missing or empty. The raw answer is still inresult. Tighten the task wording, and keep the IF guard from Step 7.
Key takeaways
- The HTTP Request node can't see JavaScript-rendered pages. A browser agent can, and in n8n it's one verified node.
- Run a task and wait + Extract Structured Data turns a sentence into JSON that Sheets, Gmail or any other node can use.
- Budget a few cents per simple run (agent plus browser and proxy), set Max Cost USD anyway, and validate the output before you write it anywhere.
From here you can build
The same five nodes work for any page without an API:
- competitor pricing pages into a weekly sheet
- job boards filtered by keyword into a morning email
- supplier or government portals checked every morning for new documents
Change the task sentence and the schema. The rest of the workflow stays the same.
I tested Browser Use's cloud browser and hosted agents in more depth in my Browser Use review, including stealth checks and model comparisons. If you're choosing a provider, start with my comparison of the best browser automation APIs for AI agents.
Your n8n workflows already know what to do with data.
Now they can go and get it.
Looking for technical content for your company? I can help — LinkedIn · kevinmenesesgonzalez@gmail.com





![Quadruped Robot Software — Ch.10: Quadruped AI and Physical AI [KR]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ffiibwtdu8wij6gnnuoyc.png)

