How to Pack Public File URLs Into a Checksummed ZIP for Catalog Work

Catalog imports, vendor handoffs, and offline reviews often start from a spreadsheet of file addresses you already have: product images you are licensed to reuse, public PDF spec sheets, a CSV or two. The messy part is not finding the URLs. It is fetching them one by one, noticing that one link returned an HTML page, and handing someone a folder of unnamed partial files.

This walkthrough uses SteadyScrape’s Bulk Image and File Downloader. You supply a short list of public direct file URLs. The Actor fetches the files that actually download, packs them into one ZIP, and writes a status row for every URL with the byte count, detected type, SHA-256, and a sanitized source URL. The same manifest is stored as JSON and inside the ZIP.

The Actor does not crawl, search, follow page links, render HTML, or log in. It does not read cookies, tokens, or custom headers. It does not bypass DRM or paywalls, unpack archives, or execute anything it downloads. Public access is not a copyright license. You need the right to fetch and reuse each file. This Actor does not scan for malware and does not clear copyright.

The buyer problem

If you already have the file addresses, you want one bounded pack that answers:

• Which URLs were actually saved?
• What type was detected from the bytes, not just the filename?
• What is the SHA-256 of each saved file?
• Which URLs were HTML pages, private addresses, or credential links, and therefore left out?

A site crawler is the wrong tool. So is pasting signed URLs that embed tokens into a dataset you will store. This Actor is for direct public files you are allowed to fetch.

SteadyScrape publishes a separate public Actor for turning a page into JSON: AI Page To JSON. That is a different job. This downloader only retrieves the file URLs you supply.

What the downloader returns

The dataset has one row per processed URL. The default key-value store has three fixed keys: OUTPUT (counts only), MANIFEST (the full JSON manifest), and PACK (the ZIP, when it fits). File bytes are not copied into the key-value store a second time. Keys are not built from the source URLs.

saved and saved_type_mismatch are the only statuses that put a file in the ZIP. A row is not labeled as a successful image when the body is an HTML page.

How to run it in Apify Console

1. Open Bulk Image and File Downloader on the Apify Store.
2. Paste up to 20 direct http or https file URLs. Use port 80 for http and port 443 for https.
3. Leave the caps at their defaults unless you want a smaller run. You can lower maxUrls, maxFileBytes, maxTotalBytes, maxConcurrency, maxRuntimeSeconds, maxRedirects, connectTimeoutSeconds, and readIdleTimeoutSeconds. You cannot raise them.
4. Run the Actor.
5. Read the dataset for per-URL status. Download the PACK record for the ZIP and MANIFEST for the JSON copy.

There is no field for headers, cookies, proxy, a custom DNS resolver, localhost, or a fixture path. A URL with userinfo, a nonempty credential query value (token, signature, api_key, password, and similar names), or a fragment that carries token material is rejected and is not written to the dataset, manifest, or log.

Example input using two well-known public files (a W3C PNG and the RFC 2606 text file):

{ "urls": [ "https://www.w3.org/Icons/w3c_home.png", "https://www.rfc-editor.org/rfc/rfc2606.txt" ] } 

The Store page's example input downloads three small public files (this PNG, a W3C test PDF, and the RFC text file), so a first run shows a real ZIP and manifest.

How to call it from Python

This uses the official Apify Python client and the Actor's real input fields:

import os from apify_client import ApifyClient client = ApifyClient(os.environ["APIFY_TOKEN"]) run = client.actor("steady-scrape/bulk-file-downloader").call( run_input={ "urls": [ "https://www.w3.org/Icons/w3c_home.png", "https://www.rfc-editor.org/rfc/rfc2606.txt", ] } ) for row in client.dataset(run["defaultDatasetId"]).iterate_items(): print( row.get("status"), row.get("saved"), row.get("sourceUrl"), row.get("detectedMediaType"), row.get("byteLength"), row.get("sha256"), row.get("zipEntryName"), row.get("errorCode"), ) 

Then download PACK and MANIFEST from the run’s default key-value store.

Output examples

Measured sample

A test run fetched the W3C PNG and RFC 2606 text file above, plus three URLs that should not be saved: an HTML page, a non-public address, and a credential URL. Two files were saved. The HTML page was rejected. The credential URL was omitted from sourceUrl.

Measured saved PNG row (the request followed one public redirect to the current W3C asset URL):

{ "index": 0, "status": "saved", "saved": true, "partial": false, "sourceUrl": "https://www.w3.org/Icons/w3c_home.png", "sourceUrlRedacted": false, "finalUrl": "https://www.w3.org/assets/logos/w3c-2025-transitional/w3c-72x48.png", "finalUrlRedacted": false, "httpStatus": 200, "declaredContentType": "image/png", "detectedMediaType": "png", "typeMatch": "match", "byteLength": 1848, "sha256": "590608bf256237f22d84bbab2a0a1afc0d4f20db6350bbac2c6841f1f3e13b01", "zipEntryName": "w3c-72x48.png", "errorCode": null, "redirectCount": 1 } 

Measured saved text row:

{ "index": 1, "status": "saved", "saved": true, "partial": false, "sourceUrl": "https://www.rfc-editor.org/rfc/rfc2606.txt", "httpStatus": 200, "declaredContentType": "text/plain", "detectedMediaType": "text", "typeMatch": "match", "byteLength": 8008, "sha256": "b6869c8984701701bc2e6973b6ffc750d497f845cc1a65a106e9301590a13ab0", "zipEntryName": "rfc2606.txt", "errorCode": null, "redirectCount": 0 } 

Measured non-success outcomes from the same run:

• https://www.iana.org/help/example-domains returned HTTP 200 with declaredContentType text/html and detectedMediaType html. status was rejected_disallowed_type. The file was not added to the ZIP (zipEntryName was null, partial was true).
• A non-public address was rejected_nonpublic_address, with sourceUrl null and no bytes saved.
• A credential URL was rejected_credentials, with sourceUrl null and sourceUrlRedacted true. The secret URL was not stored.

Trimmed OUTPUT fields from that run: packKey PACK, manifestKey MANIFEST, packBytes 6262, savedCount 2, failedCount 3, omittedUrlCount 0, finishedReason completed. The ZIP contains the two saved files plus manifest.json.

Those checksums describe the bytes stored in that run. If a publisher replaces a file, a later run can produce a different sha256.

What it costs

The Actor uses pay-per-event pricing:

• $0.004 per saved file (a file that actually lands in the ZIP).
• $0.003 when a run includes at least one saved file.

Worked examples:

Rejected HTML pages, private addresses, and credential URLs are not saved files under this model.

With pay-per-event pricing these events are the whole Actor price; there is no separate start fee and no compute charge on top. A run that saves nothing costs nothing. The Store page shows the same event list.

Limits and allowed types

Defaults are the hard maximums. Input can lower them and cannot raise them.

Saved types are JPEG, PNG, GIF, WebP, BMP, TIFF, PDF, plain text, CSV, JSON, ZIP, gzip, 7z, RAR, DOCX, XLSX, PPTX, and legacy OLE Office files when the name or content type says so. HTML, SVG, JavaScript, scripts, WASM, and executables are rejected. An unknown binary is rejected. If the declared type and the sniffed type disagree and either side is an active rejected type, the file is not saved.

Only public http and https are allowed. Private, loopback, and other non-public addresses are rejected before they are used. Archives that are allowed are stored whole; they are not extracted.

manifest.json is reserved inside the ZIP. Entry names are sanitized, and collisions get a numeric suffix.

Scheduling and integrations

A run is a normal Apify Actor run. You can start it from Console, the Apify API, or an Apify schedule (for example, to keep dated, hashed copies of the same public files every week). Apify's Make, Zapier and n8n integrations and the Apify MCP server call Actors through that same API. The Actor itself does not create a webhook receiver or expose its own MCP endpoint, and standby mode is not enabled.

FAQ

Does it crawl a site or discover files?
No. You provide each direct file URL.

Can it use a signed or expiring link?
Query names such as token, signature, X-Amz-Signature, and X-Goog-Signature are rejected so those secrets are not stored. Use a URL you are allowed to fetch that does not put a credential in the query or fragment.

What if the URL is an HTML page?
The body is not treated as a saved image or document. In the measured sample, an HTML page was rejected_disallowed_type and left out of the ZIP. That is type rejection, not a general challenge-page detector.

Will it open a ZIP it downloaded?
No. Archive bytes are stored. They are not extracted, and nothing downloaded is executed.

Are the files checked for malware or copyright?
No. You must be entitled to fetch and reuse the files. The Actor does not grant that right and does not claim the files are safe to open.

Does a failed URL still produce a status row?
Yes. The row records the failure and any bytes already read. Other URLs keep the status they earned. Only saved files are charged ($0.004 each, plus $0.003 once per run that saves at least one file). A rejected URL is not a saved file.

Try it

Open Bulk Image and File Downloader on the Apify Store, keep the example input, and run it. You get a ZIP with three small public files, a MANIFEST with their SHA-256 checksums, and one status row per URL.

If you need text out of the PDFs you downloaded, PDF to Text Markdown Extractor is the companion Actor.