Python testing with pytest. Some quick tips
When dealing with larger suites (> 1,000 tests runs including variants) with pytest, it can become hard to focus and reason about specific failures.
Specific failures are what you need to deal with when re-factoring code, so you are not swept away by the waves of errors that non-trivial changes can lead to. Even 1-5 line code changes might propagate a legion of fiesty integration problems in the test suite.
- -k flag is your friend, which helps you to select tests and get limited feedback on each thing you care about
- --pdb flag may help to debug failing cases and prevent iteration fatigue and exhaustion if you get 97 failures
- If assertion error messages are unhelpful, please try to improve them.
Nothing is worse than a truncated test log, or combing several hundred KB of test log, looking for a good anchor point.
If testing code is unclear, then it's not documentation. Try to improve it.