Managing Test File Fixtures: Naming, Manifests and Generating Them in CI

Test files tend to accumulate. Someone adds big.pdf for a bug, someone else adds big2.pdf, and a year later the repository carries 400 MB of binaries nobody can explain. This post describes the setup we use instead: small committed templates, a manifest that says what each fixture is for, and helpers that build large files at test time.

1. Name fixtures after what they test

A fixture's name should tell you its type, its exact size and why it exists. We use:

<rule>-<edge>-<bytes>.<ext>

max-size-under-10485759.pdf
max-size-exact-10485760.pdf
max-size-over-10485761.pdf
type-png-as-jpg-2048.jpg
empty-0.pdf

When a test fails, the file name in the log already tells you the boundary it was probing. Compare that with test2.pdf.

2. Keep a manifest with expected results

The single most useful habit: every fixture has a line in a manifest that says how it is produced and what the system should do with it. Tests read the manifest instead of hard-coding expectations.

[
  {
    "file": "max-size-over-10485761.pdf",
    "bytes": 10485761,
    "source": "/generator?ext=pdf&bytes=10485761",
    "expect": { "status": 413, "code": "FILE_TOO_LARGE" }
  },
  {
    "file": "type-png-as-jpg-2048.jpg",
    "bytes": 2048,
    "source": "valid PNG renamed to .jpg",
    "expect": { "status": 422, "code": "TYPE_MISMATCH" }
  }
]

The source field is what makes this reproducible. A generator link with an exact byte count, like the one above, lets anyone rebuild the same file without asking where it came from. The shareable presets playbook covers how to organise those links for a team.

3. Do not commit large binaries; build them

Git keeps every version of every binary forever. A 100 MB fixture that changes twice costs 300 MB in every clone. Our rule is: commit templates under about 100 KB, and generate anything larger in the test setup.

For formats that tolerate whitespace, padding is trivial. JSON allows whitespace after the last token, so this produces a valid file of an exact size:

import { writeFileSync } from 'node:fs';

function padJson(obj, targetBytes) {
  const body = Buffer.from(JSON.stringify(obj), 'utf8');
  if (body.length > targetBytes) throw new Error('template larger than target');
  return Buffer.concat([body, Buffer.alloc(targetBytes - body.length, 0x20)]);
}

writeFileSync('payload-750000.json', padJson({ items: [] }, 750000));

Binary formats need padding in a place the parser ignores. PNG is a good example because its structure is simple: a signature followed by chunks, each with a length, a four-letter type, data, and a CRC-32. Any chunk whose type starts with a lowercase letter is "ancillary" and may be skipped by decoders. So we insert one private ancillary chunk just before the final IEND chunk:

import { readFileSync, writeFileSync } from 'node:fs';
import { crc32 } from 'node:zlib'; // Node 20.15+ or 22.2+

function padPng(png, targetBytes) {
  const overhead = 12; // 4 length + 4 type + 4 CRC
  const dataLen = targetBytes - png.length - overhead;
  if (dataLen < 0) throw new Error('target must be at least template + 12 bytes');

  const iendAt = png.length - 12; // IEND is always the last 12 bytes
  const typeAndData = Buffer.concat([Buffer.from('paDd', 'ascii'), Buffer.alloc(dataLen)]);
  const len = Buffer.alloc(4); len.writeUInt32BE(dataLen);
  const crc = Buffer.alloc(4); crc.writeUInt32BE(crc32(typeAndData));

  return Buffer.concat([png.subarray(0, iendAt), len, typeAndData, crc, png.subarray(iendAt)]);
}

const template = readFileSync('fixtures/templates/1x1.png');
writeFileSync('max-size-over-10485761.png', padPng(template, 10485761));

The result opens normally in image viewers and passes signature checks. We verify it once with pngcheck or by decoding it with the same image library the server uses. The same idea (pad in a region the format defines as ignorable) applies to other formats, but the details differ per format, which is why we lean on the generator for the rest and keep its links in the manifest.

When a test only checks size and the server does not look at content, a raw file is enough and much faster to make: truncate -s 10485761 over.bin on Linux, mkfile -n 10485761 over.bin on macOS, or fsutil file createnew over.bin 10485761 on Windows.

4. Pin contents with hashes

Store a SHA-256 next to each committed template, and check it in CI. If a template changes, every size test built from it changes too, and you want that to be a visible diff, not a surprise.

sha256sum fixtures/templates/* > fixtures/templates.sha256
sha256sum -c fixtures/templates.sha256

5. Separate fixture types by purpose

• Dummy files for limits, types and transport. Generated, content-free, safe to share anywhere.
• Real samples for rendering, OCR, media and compression tests. Chosen by hand, licensed for the purpose, and small in number.
• Never production data. Customer uploads copied into a test bucket are a data protection incident waiting to happen, and they make test failures impossible to share in a public bug tracker.

6. Match the size mix to reality, then add the edges

If you have upload metrics, look at the size distribution by type. Your everyday fixtures should resemble the common case (for many apps that means images of a few hundred KB to a few MB), because that is where performance regressions hurt most. Then add the boundary files on top. Do not skip the boundaries because they are rare; they are rare in traffic and common in support tickets.

7. Clean up after the run

Every upload test writes to storage. Use a dedicated test account or a filename prefix, and add an afterAll hook or a scheduled job that deletes anything with that prefix older than a day. Also delete generated local files in CI so the workspace cache does not grow.

Checklist

1. Descriptive names with rule, edge and byte count.
2. A manifest with source and expected result per fixture.
3. Templates under ~100 KB in Git, larger files built at test time.
4. Hashes for committed templates.
5. Dummy files and real samples kept apart; no production data.
6. Cleanup of both remote storage and local build output.

For wiring these fixtures into browser tests, see automating upload tests with Playwright, Cypress and Selenium.