TXT, CSV, JSON and PDF Test Files: What Each One Actually Exercises

When a ticket says "test the upload with a large file", the format matters more than people expect. A 20 MB TXT file, a 20 MB CSV and a 20 MB PDF go through the same upload endpoint but usually end up in completely different code: a text preview, an import job, a document renderer. This post covers what is inside each of the four most common test formats, what happens when they are padded to a specific size, and which bugs each one tends to find.

TXT: no structure, all encoding

Structure: none. There is no header and no magic number, which is why file --mime-type has to guess (text/plain, sometimes with a charset).

What it is good at testing:

• Bytes versus characters. In UTF-8, "a" is 1 byte, "é" is 2 bytes and many emoji are 4 bytes. A limit described as "10,000 characters" and enforced as 10,000 bytes rejects text that looks short enough. Test with a file that is under the limit in characters but over it in bytes.
• Byte order marks. A UTF-8 BOM is the three bytes EF BB BF at the start of the file. Some tools add it, many parsers do not strip it, and it shows up as invisible junk at the start of the first line.
• Line endings. \r\n versus \n changes line counts and can leave a stray carriage return on each value.
• Preview truncation. Does the app try to render a 50 MB text file in a single DOM node? It should show the first part and offer a download.

When padded: our text formats are padded with spaces, so a padded TXT is mostly whitespace. That is fine for size and transport tests. It compresses extremely well, so do not use it to measure realistic transfer times over gzip.

CSV: structure by convention

Structure: rows of delimited fields, loosely standardised by RFC 4180 (comma separator, CRLF line endings, double quotes around fields that contain commas, quotes or newlines, and "" for a literal quote). In practice every spreadsheet program has its own dialect.

What it is good at testing:

• Quoting. A field like "Smith, Jane" must stay one field. A field with an embedded newline must stay one row.
• The Excel BOM. Files saved as "CSV UTF-8" from Excel start with a BOM, so the first header becomes \ufeffid instead of id and a lookup by column name fails. This is one of the most common import bugs we know of.
• MIME type confusion. Browsers take the Content-Type for a local file from the operating system. On Windows machines with Excel installed, a .csv often arrives as application/vnd.ms-excel rather than text/csv. A server that only allows text/csv rejects valid files from those users.
• Memory. Does the importer stream rows or read the whole file into memory first? A CSV of a few hundred MB will tell you quickly.

When padded: the generator's CSV template is a header and a few rows, padded with spaces. Check how your importer treats trailing whitespace: some parsers ignore it, some produce an extra empty or whitespace-only row. That is a legitimate test in its own right. If you need many realistic rows rather than padding, generate them:

import csv

target = 520_000  # bytes, approximately
with open('import.csv', 'w', newline='', encoding='utf-8') as f:
    w = csv.writer(f)
    w.writerow(['id', 'email', 'name', 'note'])
    i = 0
    while f.tell() < target:
        w.writerow([i, 'user%d@example.test' % i, 'User %d' % i, 'has, a comma'])
        i += 1
print(i, 'rows')

This gets close to the target; for an exact byte count use /generator?ext=csv&bytes=520000. The large CSV import benchmarking scenario covers timing an importer with both kinds of file.

JSON: strict syntax, big memory

Structure: a strict grammar (RFC 8259). Unlike CSV, there is exactly one correct way to parse it, so malformed JSON fails loudly.

What it is good at testing:

• Body size limits. Framework defaults are small. Express's express.json() rejects bodies over 100 KB with a 413 unless you raise limit. Send 99 KB, 100 KB and 101 KB payloads and check the error is handled, not surfaced as a crash.
• Parse cost. JSON.parse on a large string blocks a Node.js event loop for the whole parse, and the resulting object takes several times the file size in memory. A 50 MB JSON upload is a good way to see whether an endpoint should be using a streaming parser.
• Client rendering. A frontend that renders every item of a large JSON response without virtualization will stall. Oversized mock responses find this early; see JSON payload bloat for REST clients.

When padded: JSON allows whitespace between tokens, so a space-padded JSON file still parses to the same small object. That makes it ideal for testing size limits separately from parse cost: the transport sees 5 MB, the parser does almost no work. If you need parse cost too, generate a large array of real objects.

PDF: a container with a lot of machinery

Structure: a PDF begins with %PDF- followed by a version (bytes 25 50 44 46 2D), contains numbered objects, a cross-reference table giving the byte offset of each object, and ends with %%EOF. Readers find the cross-reference table by reading backwards from the end of the file.

What it is good at testing:

• Size limits on a type users really upload. PDFs are the most common "document" upload, and our padded PDFs remain valid, so they pass type checks and still open.
• Post-processing. Thumbnailing, page counting, text extraction and virus scanning all run on PDFs. A large, valid, single-page PDF checks that these steps handle size without also testing heavy rendering.
• Byte integrity. Because readers use byte offsets, a PDF that has lost or gained bytes in transit often fails to open or triggers a "repair" prompt. Compare hashes after upload.

To inspect a PDF from the command line: pdfinfo dummy.pdf (from poppler-utils) shows page count and file size, and qpdf --check dummy.pdf reports structural problems.

What it is not good at: a padded one-page PDF will not stress a renderer the way a 300-page scanned document with embedded fonts does. For OCR or rendering performance, use real documents.

Side by side

Format	Signature	Common MIME	Best for
TXT	none	text/plain	Encoding, previews, raw size
CSV	none	text/csv (varies by OS)	Importers, quoting, BOM handling
JSON	none	application/json	Body limits, parse cost, client rendering
PDF	25 50 44 46 2D	application/pdf	Document uploads, post-processing, integrity

Per-format details live on the format pages: TXT, CSV, JSON and PDF. For any other extension, MIME lookup tells you the expected type.