Tests that try many inputs to break your rules
Tell Claude the rules your code must always follow, and it writes Python tests that throw many generated inputs at each rule. These tests can find odd cases you would never list by hand. Claude writes the test file and you run it.
How to use it
Ask Claude in plain English, for example:
generate property testsmake hypothesis tests for slugifyproperty-test these invariantsfuzz-test this function
Ask in plain English at any time, on any level (Light, Medium or Deep). Point it at a list of rules or at a spec audit file. TLM Forge also tells Claude to write these tests on its own when a spec audit lists rules that have none, and to suggest them when you change sign-in and permissions (auth), encryption, personal data or payments code.
Words you will see on this page
- Property test
- A test that states a rule for every valid input and lets a tool try many inputs to break it.
- Invariant
- A rule your code must always follow, such as "the total is never below zero".
- Hypothesis
- A Python library that generates test inputs and, when a test fails, shrinks the failing input to a simpler one.
- @given
- The line in a Hypothesis test that says which kinds of inputs to generate.
- Spec audit (spec_audit.md)
- TLM Forge's written check of a feature's requirements, which can list the rules the code must follow.
Example tests vs property tests
| Compare | Example test | Property test |
|---|---|---|
| What you write | One input and the answer you expect | A rule that must hold for every valid input |
| Inputs tried | Only the ones you listed | Many generated ones: up to 200 by default, 1000 for security-critical code |
| Odd cases | Found only if you thought of them | Can reach odd Unicode characters and edge values you did not list |
| When it fails | Shows the case you wrote | Shows the input that broke the rule |
When it fits, and when it does not
| What you hand it | What Claude does |
|---|---|
| A clear rule: for every valid input, the output meets a condition | Writes one property test for that rule |
| A "rule" that is really one specific case | Pushes back and suggests an ordinary example test |
| A rule about a sequence of calls, such as IDs that keep increasing | Writes a step-by-step state machine test instead |
| A click-through UI flow | Says it belongs to end-to-end testing tools |
| Behaviour with no visible result, such as "always sends an email" | Says there is nothing to check and suggests an example test |
What it does
Most tests check examples you picked: given this input, expect that output. A property test works differently. You state a rule that must hold for every valid input, and a library generates many inputs to try to break it. This feature turns your plain-English list of rules into such tests. It writes Python using the Hypothesis library, with one test per rule, so a failure points straight at the rule that broke.
TLM Forge tells Claude to be strict about quality. Each rule must have the shape "for every input that meets a condition, the output meets a condition". If a rule does not fit, Claude pushes back and suggests a normal example test. Generated inputs look like real data, such as IDs, emails and short text. Every list and text has a size limit so tests stay fast, and every failure message shows the input that caused it.
It is honest about where these tests do not help. Click-through UI flows belong to end-to-end tools. Tiny functions are clearer with example tests. Behaviour with no visible result, such as "always sends an email", has nothing to check. Rules about a sequence of calls, such as IDs that keep increasing, get a step-by-step state machine test instead.
How it works, step by step
Turn your rules into for-every statements
Claude reads your list or spec and rewrites each rule as "for every valid input, this holds". Rules that are really single cases get pushed back.
Pick realistic inputs
Each input gets a generator that matches real data. Every one has a size or value limit, so failures shrink quickly.
Set how many inputs to try
Each test tries up to 200 inputs by default, or up to 1000 for security-critical code.
Write one test per rule
One Python file per feature. The top of the file names the spec it came from, and each rule gets its own test whose failure message shows the input.
Save it where pytest finds it
The file goes in a property folder inside your tests folder, such as tests/property/test_coupons_invariants.py. During a /tlmforge:build run, the path is also noted in the feature's status file (STATUS.md).
You run it
Claude does not run the tests. You run them with pytest and read any failing input Hypothesis reports.
What it looks like
A sample generated test file (invented example). It checks one rule, "the total is never below zero", against up to 1000 generated carts, and a failure prints the exact cart and coupon that broke it.
"""Property-based tests for coupons. Generated by property-test-generator skill.
Each test corresponds to an invariant in specs/coupons/spec_audit.md."""
from hypothesis import given, settings, strategies as st
from shop.cart import apply_coupon
@given(items=st.lists(price_strategy, max_size=100), coupon=coupon_strategy)
@settings(max_examples=1000)
def test_total_never_below_zero(items, coupon):
total = apply_coupon(items, coupon).total
assert total >= 0, f"Invariant violated: {items=!r} {coupon=!r} gave {total=!r}"Why it is better
Finds cases you did not think of
Without TLM Forge, your tests cover only the cases you wrote down. With it, generated inputs can reach the odd Unicode character or edge value that quietly corrupts data.
Each failure names the broken rule
Without it, a failing test points at a line of code. With it, each rule has its own test and the file names its source spec, so you know which rule broke.
Failures you can read
Without it, an assertion can fail with no hint of the input. With it, every failure message must include the inputs, so you see exactly what broke the rule.
Pushes back on vague rules
Without it, a fuzzy rule becomes a fuzzy test. With it, Claude asks for a clear condition first, because a rule that cannot be tested is a sign the spec is unclear.
Good to know
- Python only, using the Hypothesis library. No other language is supported.
- Claude writes the tests but does not run them. You run them with pytest.
- Not for UI click flows, tiny functions, or behaviour with no visible result. Claude says so and suggests an example test instead.
- Each test tries up to 200 inputs by default and up to 1000 for security-critical code, and every input generator has a size or value limit.