What do I need from a code analysis tool
I am a fan of static code analysis tools,
-Wall, etc. Ignoring advice of people who professionally work with programming languages isn't wise. But besides warnings themselves, the linter must actually be convenient to use.
So here's a list of highly advisable features for a linter:
- Every single warning type must be toggleable. If something is an error for sure, ideally the compiler can tell. Otherwise there must be an option to disagree with the tool.
- Every warning must be able to be disabled locally. If one file or function must use - with precautions - a debatable mechanism, it's not a reason to allow it globally.
- Each warning must be ignored separately. One
#noqa for everything isn't enough.
- If a warning is disabled locally but wasn't raised, this must be a separate message. Perhaps the author tried to work around a bug in an old version of a linter, and it's not needed anymore?
- If the check produced warnings, the exit code must be nonzero.
- The ideal check state is zero warnings. But if the linter is run on pre-existing code, it must tell whether there are new warnings compared to a known baseline state. This allows to fix the code gradually, without introducing new defects. Of course, if there are less warnings than before, the baseline state must be updated.
Some of these items are trivial, yet I have seen a counter-example for each. Everything except the last point doesn't even require special tricks, so please, if you somehow find yourself writing a linter - keep them in mind.