Property-Based Testing

Status: Complete
Category: Testing
Default enforcement: Advisory
Author: PushBackLog team

Tags

• Topic: testing, quality
• Skillset: backend, frontend, fullstack
• Technology: generic
• Stage: execution

Summary

Property-based testing generates hundreds or thousands of random inputs and verifies that specified properties hold across all of them, rather than asserting specific expected outputs for handpicked inputs. It excels at finding edge cases that example-based tests miss — boundary values, unexpected combinations, and inputs that fall outside the assumptions encoded in the test author’s mental model. It complements example-based testing rather than replacing it.

Rationale

Example-based tests are limited by the test author’s imagination

A conventional unit test asserts that parseDate("2024-01-15") returns the expected date. The test author writes a few representative examples and considers the function tested. But the test author’s examples encode their assumptions about valid input — they write tests for input they expect to work, and perhaps a few error cases. They don’t test parseDate("2024-02-30"), parseDate(""), parseDate("00/00/0000"), or parseDate("2024-01-15T00:00:00Z") unless they think to.

Property-based testing removes the “think to” requirement. The framework generates inputs from the specified domain and tries them all — including the ones the test author would not have tried.

Properties describe invariants that always hold

Instead of “given this input, expect this output”, property-based tests specify invariants: “for any valid date string, parsing and re-serialising should produce the original string” or “for any two numbers, add(a, b) should equal add(b, a)”. These invariants describe the essential behaviour of the function independently of specific examples.

Guidance

Core concepts

TypeScript example (fast-check)

import fc from 'fast-check';

// Property: reversing a string twice returns the original string
test('double reverse is identity', () => {
  fc.assert(
    fc.property(fc.string(), (s) => {
      expect(reverseString(reverseString(s))).toBe(s);
    })
  );
});

// Property: parsing a valid ISO date and re-formatting always produces the original
test('parse-format round-trip', () => {
  fc.assert(
    fc.property(
      fc.date({ min: new Date('2000-01-01'), max: new Date('2099-12-31') }),
      (date) => {
        const iso = date.toISOString().split('T')[0];
        const parsed = parseDate(iso);
        expect(formatDate(parsed)).toBe(iso);
      }
    )
  );
});

// Property: addition is commutative
test('add is commutative', () => {
  fc.assert(
    fc.property(fc.integer(), fc.integer(), (a, b) => {
      expect(add(a, b)).toBe(add(b, a));
    })
  );
});

Common generators (fast-check)

fc.integer()           // whole numbers
fc.float()             // floating point
fc.string()            // arbitrary strings
fc.emailAddress()      // valid email addresses
fc.uuid()              // UUIDs
fc.array(fc.integer()) // arrays of integers
fc.record({            // structured objects
  id: fc.uuid(),
  amount: fc.nat(),
  currency: fc.constantFrom('GBP', 'USD', 'EUR'),
})
fc.oneof(fc.string(), fc.integer(), fc.boolean()) // union types

Shrinking in practice

When a failure is found, fast-check automatically tries to find the smallest input that still fails:

Property failed after 47 tests
  seed: -1982731645
  counterexample: ["", -1]
  Shrunk 3 times from: ["hello world", -9234]

The counterexample is the minimum reproducing case — much simpler to debug than the original random failure.

When property-based testing adds most value

Combining with example-based tests

Property-based testing does not replace example-based tests. Document behaviour intent with example-based tests ("given a trial user, returns false for billing features"). Use property tests to explore the input space and verify invariants across all inputs.