How I write 1000s of tests with little effort: • With property-based testing What is it, and why you should care: With normal unit tests, it's nearly impossible to cover all input combinations for your tested code. Writing 1000s of separate tests for all inputs is both expensive and time-consuming But property-based testing fixes this. It's a type of testing to check if our system holds a property when tested with a large number of inputs. Such property can be a business rule. As an example, think about a function that calculates the shortest path between two cities. For this, we can define properties like: • Symmetry: The shortest path from A to B should be the same as from B to A. • Non-Negativity: The length of the shortest path should never be negative. To execute property-based testing, we do the following: 1. Define a property 2. Generate a large number of input combinations 3. Run the test logic for each 4. Check if the property holds for each If a test fails, you have a bug in your system representing an edge case. To generate tests, you can use property-based testing libraries. You can find them in any language. Bugs don't hide in the happy paths. They hide in the edge and corner cases. By using property-based testing, you can build confidence. Don't write many tests. Generate them. ----- To learn more about Property-Based Testing, check out my latest newsletter post: craftbettersoftware.com
In Kotlin/Java, we use Parameterized tests. I haven't looked into libraries for generating the different inputs tho. It is a great thing to investigate. Thanks for the inspiration Daniel Moka
Thanks for the post! Which property-based testing library do you recommend for someone just getting started?
Did you write an article about this with some examples?
I need to look into this! Feels like exactly what’s unit testing SHOULD be.
This is brilliant, Daniel. This is something new to me. Is it a library or something?
Oh, examples in the article. Saved for latter today Daniel Moka
Such a brilliant approach to bullet-proof the system against different input values. Very informative, Daniel. 👌
Good approach Daniel Moka! Currently struggling with writing tests for a large transport network optimisation. Just to many cases that could go wrong to cover them all directly. Might be useful here.
In many languages, you can also generate tests programmatically, for example by iterating over an array of values (or even a CSV file if a volume of various examples is big). This is kinda similar to property testing in this regard that you can easily test a huge amount of cases without making your suite verbose. The main difference is that you still provide all the values manually (which requires more effort but gives more precise control). Have you also tried this method? How would you compare it to property-based testing where values are generated by a library?
I help you master Test-Driven Development (TDD)
5moAnd no, AI won't solve this. Using AI, you could generate 1000 of tests, each with 50 lines, resulting in a total of 50,000 lines to maintain. However, with property testing, you can generate thousands of tests with just 50 lines in total. Choose your weapon.