Python - Pytest and ContextLib
Earlier this year, I got a contribution merged to pytest. I was really happy about this as it solved a confusing error message, and was to a project I use and care about.
Software testing is so important. Without it refactoring complex code becomes more stressful and risk-prone than needed.
Noise while testing can reduce the benefits of tests. Be that unclear messages, or confusing details in tracebacks.
When using standard library hurts test readability
The feature I worked on was around asserting raised errors. I understand the complex and important work the Pytest team do, but also sometimes that does not result in the most optimum API, or supporting code. I looked through how the code worked, and set about a rewrite.
Coming from private industry, I get to use newer python features often. I guess this is a privilege I had overlooked. The only thing that was not working as expected or desired in my own assert_raises implementation, was that context manager internals were showing up. I wanted to know why Pytest did not use contextmanager, but also how they suppressed their own wrapper code.
Great code & documentation
It did not take much digging to find that utility code which is generic and not meant to be surfaced can be omitted from tracebacks using a decorator. This is a real credit to the pytest team, their code and documentation. The below was all I needed.
__tracebackhide__ = True
I have not upstreamed my changes to contextlib. I Feel it might even be inappropriate to do so. I think being a good OpenSource community member means sometimes understanding that it is okay that code does not resonate or meet one coders exact standards, or the way I would do things. Changing contextlib to be aware of a testing library is kind of a smell. I would love to find an alternative way to add core contextlib to a blocklist, in a way that meets the runtime requirements of all testing libraries, including pytest.
I wish there were a PEP standard for omitting logic that is tested externally, like contextlib. I am not aware of one, but it might exist. If you know of it. Please let me know!
- OpenSource code needs contributors.
- Error messaging is important.
- Reducing noise is part of improving error messages.
- Standard library can really help shorten code.
- consider all the inputs before suggesting a change.