Testing a CPAN module that you're authoring

specific files found in a release tarball

• t/*.t
  • can be run using perl Makefile.PL; make; make test — see ExtUtils::MakeMaker §make-test
  • can be run using perl Build.PL; ./Build; ./Build test — see Module::Build §test
  • can be run using prove -l — see prove
  • can be run using perl -Ilib t/foobar.t
• test.pl
  • more often found in older releases; if it exists, this should be used to run the tests
• META.json
  • if it includes a section called 'rules', then it will be interpreted by TAP::Parser::Scheduler

related modules (central ones)

There are innumerable testing modules, but some of the really central ones are:

• Test::Harness, included in Perl core since 5.000
  • TAP::Harness, the apparent successor to Test::Harness, included in Perl core since 5.10.1
• part of the Test-Simple distribution that was included in Perl core in 5.6.2:
  • Test::Simple (tutorial)
  • Test::More
  • Test::Builder
• pieces of Test2-Suite were included in Perl core starting in 5.25.1, as well as Test-Simple:
  • use Test2::V0, at a minimum
    • though at some point you will want to instead use Test2::V1, or more specific Test2 modules
    • more specific alternatives include use Test2::Tools::Basic
  • to learn how to use Test2, read things in roughly this order:
    1. Test2::Manual::Testing::Introduction — a basic introduction
    2. Test2::Manual::Testing::Migrating — how to migrate old tests (e.g. Test::Builder, Test::More) to Test2
       • maybe Test2::Transition — transition notes when upgrading to Test2
    3. Test2::Manual::Testing::Planning — how to set the plan (i.e. identify the number of tests that are to be run)
    4. Test2::Manual::Testing#LISTING-DEPENDENCIES — how to list dependencies
    5. Test2::Manual::Testing::Todo — how to mark tests as TODO
    6. Test2::Manual::Concurrency — how to run tests in parallel
    7. feel free to read as many pages underneath Test2::Manual as you'd like, though note that they currently total 26 files and 16,000 words
    8. finally, you could dive into all Test2 modules, though note that the currently total 117 files and the POD sections total ?? words
  • some other "how-to" guides:
    • 2019 conference presentation by Chad Granum (EXODIST), the author of Test2 and Test::Simple, 47 minutes
    • 2017 conference presentation by Chad Granum (EXODIST), the author of Test2 and Test::Simple, 49 minutes
    • 2019 article by Gábor Szabó (SZABGAB)
• others mentioned in perlmodstyle:
  • Test::Inline
  • Carp::Assert
  • Test::MockObject

For background info, see en.wikipedia.org/wiki/Test_Anything_Protocol and testanything.org.

CPANTS, a testing service for CPAN distributions

• cpants.cpanauthors.org
• they promote their concept called "Kwalitee", which is "an automatically-measurable gauge of how good your software is. That's very different from quality, which a computer really can't measure in a general sense."

particularly popular test modules

• Test::Most — most commonly needed test functions and features (38 ++s, last updated Apr 2020, first released Jan 2008)
  • Test::Deep — extremely flexible deep comparison (49 ++s, last updated Mar 2020, first released Mar 2003)
  • Test::Differences — test strings and data structures and show differences if not ok (20 ++s, last updated Jan 2022, first released Dec 2001)
  • Test::Exception — test exception-based code (38 ++s, last updated Dec 2015, first released Apr 2002)
  • Test::Warn — test methods for warnings (20 ++s, last updated Jun 2018, first released Mar 2003)
• Test::Base — a data-driven testing framework (12 ++s, last updated Apr 2018, first released Apr 2005)
• Test::Class — easily create test classes in an xUnit/JUnit style (21 ++s, last updated Feb 2021, first released May 2002)
• Test::Fatal — incredibly simple helpers for testing code with exceptions (39 ++s, last updated Aug 2020, first released Oct 2010)
• Test::Kit — build custom test packages with only the features you want (10++s, last updated Jun 2016, first released Aug 2008)
  • I swear this looks like it uses source filters, but I don't know if it actually does?
• Test::Output — utilities to test STDOUT and STDERR messages (13 ++s, last updated Feb 2021, first released Dec 2005)
• Test::Pretty — makes the output of a test script a little more... attractive? (26 ++s, last updated Sep 2015, first released Oct 2012)
• Test::Pod — check for POD errors (22 ++s, last updated Apr 2018, first released 2002)
• Test::Routine — composable units of assertion (13 ++s, last updated Jun 2021, first released Sep 2010)
• Test::TCP — test TCP-based programs (24 ++s, last updated Oct 2019, first released Aug 2008)

books about Perl testing

• "Perl Testing: A Developer's Notebook", published in 2005
• The "Extreme Perl" book, published in 2005, and entirely available for free online. Relevant chapters:
  • Chapter 6: Acceptance Testing
  • Chapter 11: Test-Driven Design
  • Chapter 13: Unit Testing
• "Beginning Perl", by Curtis "Ovid" Poe, published Aug 2012 — Chapter 14 "Testing"

articles / talks about CPAN testing

• code-maven.com/slides/perl/, by Gabor Szabo (SZABGAB), with lots of testing-specific sections: (the Wayback Machine says it was created in 2020)
  • Preface
  • Basic Testing Framework in Perl
  • Test::More
  • Test libraries
  • Mocking
  • Test::Class
  • Running and reporting tests
  • Testing networking devices
  • Database Testing
  • Microsoft Windows GUI Testing
• "Perl for Perl Newbies" (a long online tutorial, the Wayback Machine says it was most active around 2012), see section "2. Automated Testing"