Every FHIR integration team has a version of the same story. A regression in the Bundle generator ships on Tuesday. A partner reports the failure on Friday. Someone spends the weekend bisecting commits to find the culprit. The obvious fix is to catch the regression before merge, and the obvious way to do that is a test harness that validates real payloads on every commit. The mechanics are not glamorous, but they are what turns Bundle work from painful to boring.
For related developer reading, the FHIR development collection sits alongside this one on the home page. This piece is the specific build order for a harness that will actually pay off.
Start With Fixtures That Represent Real Traffic
The most common mistake is running CI validation against synthetic Bundles that no real system produces. A single Patient and one Observation is not a test fixture; it is a demo. Grab five to ten real Bundles from an integration test environment, scrub the PHI, and check them into a fixtures/bundles/ directory in the repo.
That set becomes your baseline. When a new field lands in the generator, regenerate the fixtures against a golden reference and diff. When the generator regresses, the fixture stops validating and CI turns red. If you are unsure what a validator will catch on those payloads, what a Bundle validator actually checks is the reference for the guarantee it makes.
Pin a Validator and Pin Its Profile Package
Two moving parts break test harnesses in production. The first is validator version drift. The second is profile package drift. Both look the same from the outside: yesterday CI was green, today it is red, and no code changed.
Pin both. In the repo, record the exact validator version, the profile package name, and the version of the package. Upgrade explicitly in a pull request that touches nothing else, so the diff between behaviors is easy to inspect. When a partner ships a new IG release, your PR title says exactly that.
Add a Fast Lane and a Slow Lane
A test harness that takes ten minutes to run will get skipped in local development. A harness that takes six seconds will be run without thinking.
Wire two lanes. The fast lane runs on every commit and executes only structural checks against a small subset of fixtures. It should finish inside a coffee sip. The slow lane runs on pull request open and executes the full profile-aware suite. It can take minutes because it runs less often. The site's FHIR R4 Bundle lint tool is the same shape of check that the fast lane runs, just interactive.
Decide What Turns the Build Red
Not every validator warning should break the build. Structural errors, yes. Missing required elements on a profile the receiver enforces, yes. Deprecated terminology bindings that will remain deprecated for months while the value set updates, no. Those become build artifacts, not build blockers.
Draw the line up front and put it in the harness config. Every downgrade from an error to a warning should include a link to a ticket. Otherwise the warning list grows until nobody reads it, and the harness silently becomes useless. If you want the shorthand for reading validator output while making this call, FHIR validator errors that always mean the same thing is the reference.
Handle the Transaction Bundle Gotchas Explicitly
Transaction bundles have a category of failure that structural checks alone miss. entry.fullUrl values that do not resolve inside the same Bundle. entry.request.url set to a plural resource path when a single one is expected. Circular references that only break at commit time on the receiver.
Add a transaction-specific rule set to the slow lane. It should walk every entry, resolve internal references, and confirm that Bundle.type matches the payload shape. For the specific traps that live in this space, why entry.fullUrl bites you in transaction bundles has the developer-side detail.
Publish an Artifact That Somebody Reads
A silent pass is fine. A silent fail is worse than no harness at all. Emit a short report on every run: fixtures validated, failures by severity, links to the exact FHIRPath that failed. Pipe it to the build log and attach it as a downloadable artifact.
The report is what makes the harness credible over the long run. When a partner asks how you know the outbound Bundles are valid, the answer is a report link, not a promise. The rest of the harness earns its keep by making that answer boring.
Sources
- Microsoft Tech Community institutional walkthrough on - Microsoft Tech Community institutional walkthrough on running FHIR validation inside GitHub Actions