Observing Change: A Gold Master Test in Practice - Jake Worth
Jake Worth discusses how developing legacy code is inevitable. Every production application will, at some point, become a legacy application.
In order to properly work on legacy applications, tests are a prerequisite.
- Unit tests
- API tests
- UI tests
Unit tests get the closest to specifying application behavior, but can still fall short.
Gold Master Test
Genesis of idea: Working Effectively with Legacy Code book by Michael Feathers.
"In nearly every legacy system, what the system does is more important what it's supposed to do." - Feathers
Characterization Test
A characterization test is a test that characterizes the actual behavior of the code.
Process:
- Code in test harness
- Write assertion you know will fail
- Let failure show you how to proceed
- Change the test to match the behavior
This is backwards from normal TDD, but will allow you to build test coverage for a working application for the purposes of being able to refactor code.
After this process is completed, write additional tests on a case by case basis.
Gold Master Test Process
"A gold master test is a regression test for complex, untested systems that asserts a consistent macro level behavior."
The goal of this test is to ensure that the application continues to work, even when making future changes.
Ideal Application
- It's mature
- Complex
- Expects minimal changes to output
Benefits
- Rigorous development standards
- Exposes surprising behavior
- Useful for forensic analysis
This can help give insight on the underlying behavior when adding new features in the future.
Gold Master Test Code Process
Preparation
- Get production database
- Scrub security specific data, such as emails, security codes, etc. (you can use
pg flagsfor scrubbing) - Check the cleaned up database into version control
Test Phase
- Start with empty test file
- Pipe cleaned production database into test database
- Compare current result to production database
What Happens When the Test Fails?
When the test fails:
- Check to see if the change is valid
- If valid, update the gold master database
- If not valid, reevaluate the change
Summary
The gold master test provides a high level test for an application's full functionality that includes testing against production data instead of test data, including HTML and actual database.