Some tools (older import forms, email services, spreadsheet apps) choke on big CSV files. A common fix is to split the file into smaller parts, with the header row repeated in every part. Here is a small script that does exactly that using only Python's standard library.

Why not just split -l?

The Unix split command cuts by line count. That breaks in two ways:

  • The header only ends up in the first file.
  • A quoted field that contains a line break can be cut in half.

Python's csv module reads whole records, so multi-line fields stay intact.

The script 

import csv import sys from pathlib import Path def split_csv(src, rows_per_file, out_dir="."): src = Path(src) out_dir = Path(out_dir) out_dir.mkdir(parents=True, exist_ok=True) written = [] with src.open(newline="", encoding="utf-8-sig") as f: reader = csv.reader(f) header = next(reader) writer = None out = None count = 0 part = 0 for row in reader: if writer is None or count >= rows_per_file: if out: out.close() part += 1 path = out_dir / f"{src.stem}_part{part:03d}.csv" out = path.open("w", newline="", encoding="utf-8") writer = csv.writer(out) writer.writerow(header) written.append(path) count = 0 writer.writerow(row) count += 1 if out: out.close() return written if __name__ == "__main__": if len(sys.argv) < 3: sys.exit("Usage: python split_csv.py input.csv rows_per_file [out_dir]") for p in split_csv(sys.argv[1], int(sys.argv[2]), *(sys.argv[3:4])): print(p)

Usage 

python split_csv.py products.csv 1000 parts/

This writes products_part001.csv, products_part002.csv and so on into parts/, each with at most 1000 data rows plus the header.

Quick test

I tested it with a 25-row file that included fields containing commas and line breaks, split at 10 rows. Result: three files with 10, 10 and 5 data rows, every file starting with the same header, and 25 rows in total, so nothing was lost or duplicated.

Notes

  • utf-8-sig on input strips a BOM if Excel added one; the output is plain UTF-8.
  • The file is streamed row by row, so memory use stays small even for large files.
  • If your target tool needs a BOM (some Excel versions do), change the output encoding to utf-8-sig.

If you only need the rows count per part, adjust the second argument. That is all there is to it.