Blog › Build Story · Quality
"It worked on my test file"
isn't a testing strategy.
Before anyone else touched BrandSwitch, it had to survive a real, messy document library. Here's how that testing actually happened, and what broke first.
550
real documents in the validation batch
11 min
for that batch to complete, cleanly
1,000
pages checked in another validation run
0
tidy demo files used as the real test
Why "it worked on my test file" isn't good enough for a rebrand tool
A rebrand tool's entire value proposition is consistency across a batch. That means the question that matters isn't "does it work on this one document" - it's "does it work identically on document one, document 200 and document 550 of the same run." A single tidy test file will pass almost anything. It tells you almost nothing about whether the tool holds up on the real library someone actually needs rebranded.
Building a real test set: different templates, edge cases, badly formatted legacy documents
Different templates
Policies, proposals, letterheads and training packs - not one document type repeated.
Deliberate edge cases
Multi-section documents, mixed headers, nested tables, embedded objects.
Genuinely messy files
Years of manual edits, inconsistent styles, direct formatting layered on top of everything.
What actually broke first (and it wasn't what I expected)
I expected the exotic edge cases to cause the most trouble - deeply nested tables, unusual section structures. Instead, the first real failures came from ordinary-looking documents that had quietly accumulated years of manual reformatting: a heading someone had bolded and recoloured by hand instead of using the heading style, a table where three cells had been individually overridden after a copy-paste from another file. Nothing exotic. Just the normal residue of real documents being edited by real people over time.
That was the useful lesson: the failure mode to design for isn't complexity, it's inconsistency. Real documents are inconsistent in ways a hand-built test file never is.
Before launch, the tool was validated against a real 550-document batch, not a handful of sample files.
How I validated output at scale, not just spot-checked a handful of files
Every document in the test batch got checked, not sampled. Colours, fonts, logo placement and formatting across headers, footers, tables and body text, compared against the expected result for every single file. That's slower than eyeballing a sample of ten, but it's the only way to catch the failure that only shows up on document 340 of 550 - which is exactly the kind of failure that matters most once real customers are running their own batches.
The 550-document batch as the real test, not a demo
The 550-template run that became the flagship result wasn't staged for a demo after the fact. It was the validation run itself - a real, varied document library, checked in full, that happened to also make a genuinely good story once it was done. The order mattered: it was validated first, publicised second.
What I still check before every release
The original edge-case documents didn't get archived once launch happened. They're still the standing test set every release gets checked against, alongside anything new that's turned up a surprise since. Testing on the messiest documents I can find, every time, is a permanent part of shipping this product, not a one-off step before the first customer arrived.
Quick answers
How was BrandSwitch tested before launch?
Against a real 550-document batch, not a handful of sample files - the same batch that later completed in 11 minutes. Testing also included deliberately messy, inconsistently formatted legacy documents, not just tidy templates.
What kind of documents break a rebrand tool first?
Not the obviously complicated ones. The quiet failures came from ordinary-looking documents that had been manually reformatted over years - direct formatting layered over styles, tables with mismatched cell-level overrides, and section breaks nobody remembered adding.
Does BrandSwitch get tested before every release?
Yes. Every release is checked against a standing set of edge-case documents before it ships, including the messiest legacy files collected during original testing, not just a smoke test on a clean sample.
Why not just spot-check a few documents and call it done?
Because a rebrand tool's job is consistency across an entire batch. Spot-checking a handful of files can miss an issue that only shows up on document 340 of 550, which is exactly the kind of failure that matters most in production.
Try it on your own messiest documents
BrandSwitch was validated against real, inconsistent documents before anyone else used it. It's built to handle your actual document library, not just the tidiest file in it.
Try it on your own messiest documents, not just your tidiest ones
Validated against a real batch before anyone else used it.