Perl::Tidy

It's extremely useful for fixing the indenting on huge swaths of code at once.

Guarantees

Does Perl::Tidy make any guarantees regarding the possible introduction of bugs into production code? That is — can it ever change any actual functionality, instead of just changing the presentation?

It's often said that "Only Perl can parse Perl". So, Perl::Tidy doesn't even try, instead it uses the PPI module to do the tokenizing and lexing. PPI documentation says:

"The Tokenizer uses an immense amount of heuristics, guessing and cruft ... It is by far the most complex and twisty piece of perl I've ever written that is actually still built properly and isn't a terrible spaghetti-like mess."

Perl::Tidy says:

"an error in which a reformatted script does not function correctly is quite serious. ... "

"perltidy goes to great lengths to catch any mistakes that it might make, and it reports all such errors. For example, it checks for balanced braces, parentheses, and square brackets. And it runs a perl syntax check on the reformatted script if possible. (It cannot do this unless all modules referenced on use statements are available). There is no guarantee that these checks will catch every error, but they are quite effective. For example, if perltidy were to accidentally miss the start of a here document, it would most likely report a syntax error after trying to parse the contents of the here document."

"It's difficult to give an absolute measure of reliability, but to give some practical sense of it, I can mention that I have a growing collection of perl scripts, currently about 500 MB in size, that I process in nightly batch runs after every programming change. ... Of the perl scripts that were written by people other than myself, Perltidy only fails to parse 1 correctly, as far as I can tell, and for that file, perltidy catches its own error and ends with a message to that effect."

So it's close to "yes", but I would still classify that as a "no" because Perl::Tidy has a slightly different understanding of the parse tree than Perl does.

Fallback guarantee — detect problematic modules and API calls

That's unfortunate. But is at least possible to detect whenever functionality changes actually occur, or are very likely to occur?

PPI says:

"there are going to be limits to this process. Because PPI cannot adapt to changing grammars, any code written using source filters should not be assumed to be parsable"

However, there's a caveat — Perl::Tidy has a --extended-syntax flag that allows it to handle

Some information about how to detect source filters:

• The official API includes the calls filter_add() and filter_read(), and perlfilter.pod describes how to build a source filter
• The vast majority of source filters depend on Filter::Util::Call to setup the source filtering for them. It includes Call.xs which makes the calls to filter_add() and filter_read().
  • In the same distribution is Filter::Util::exec, which does things fairly differently than Filter::Util::Call. However, I don't see many (any?) people using it — it has 0 reverse dependencies on MetaCPAN.
• A few source-filtering modules don't go through Filter::Util::Call, but instead have their own .xs file that make the calls to filter_add() and filter_read():
  • Filter::decrypt — a template for writing your own decrypt source filter, which several others copied from, such as:
  • Filter::Crypto::Decrypt
  • Filter::Rijndael
• And then there's a few that do source filtering completely differently.
  • Acme::Bleach — reads the current script into a string, munges it, and runs eval() on it
    • open 0 followed by <0> are apparently all that's need to read in the current source file
  • INABA Hitoshi's numerous source filters — these all read the current script, munge the string, write it out to another script, and then exec()s the new file

There are many tools that can be used to explore the dependency tree of installed modules — App::FatPacker::Trace, Devel::TraceUse which shows the dependency tree as... a tree, Module::ScanDeps, Devel::Modlist, B::PerlReq, etc etc.

There are even tools that can be used to explore the dependency tree of modules that aren't installed yet — MetaCPAN::Client::Release's dependency().

Fallback guarantee — compare the parse trees

The general idea is to 1) hand your Perl file to the actual Perl interpreter, have it dump the parse tree, 2) pass the file through Perl::Tidy, 3) dump the parse tree of the new version, and 4) compare the two parse trees, and flag this file as a problem if there are any differences.

TODO: look into whether it's possible to use B::Concise or OptreeCheck or B::Utils::walkallops_simple() or B::Deparse to compare the optrees.

Aha! See here — https://paperlined.org/tmp/Perl::Tidy_B::Concise.md.