Improved debugging of Lucee test cases in the build

hacking
,
1

The Lucee build by default skips any invalid tests plus some other rules

  • tests prefixes with an _
  • tests in a directory which includes an Application.cfc
  • tests which don’t extend the Lucee test BaseSpec org.lucee.cfml.test.LuceeTestCase
  • tests which don’t compile

The build uses a custom scanner for tests, which also skips any non test files and allows filtering by name or labels, i.e. ant -DtestFilter="LDEV123" or ant -DtestLabels="orm"

The must use the Lucee BaseSpec requirement is a bit annoying if you are using Testbox outside of the Lucee CI process, so I have added the ability to enable using additional baseSpecs with the Lucee build process

build: allow optionally supporting alternate test base specs, i.e “testbox.system.BaseSpec”

i.e. ant -DtestSuiteExtends="org.lucee.cfml.test.LuceeTestCase,testbox.system.BaseSpec"

I noticed this was annoying when I added some CI for the lucee-mariab db extension

But skipping invalid tests can be annoying / frustrating when developing or debugging tests, so we have a few build flags to make your life easier as a developer, hacking away, trying to making Lucee even better!

If you run with ant -DtestDebug="true" you will see a list of all the CFCs under the test directory and why they weren’t run. If you pass in -DtestDebugAbort="true" the build will exit after scanning thru all the tests, so you don’t need to wait or ctrl-c to abort the build, as it takes a few minutes to run all our wonderful tests.

I’ve added a Github Action Workflow, so you can try this out on your forked repo

The output looks like this

     [java]    [script] -------------- Start Tests -----------
     [java]    [script] /home/runner/work/Lucee/Lucee/test/components/_Administrator.cfc :: test has _ prefix (_Administrator.cfc)
     [java]    [script] /home/runner/work/Lucee/Lucee/test/_setupTestServices.cfc :: test has _ prefix (_setupTestServices.cfc)
     [java]    [script] /home/runner/work/Lucee/Lucee/test/_testRunner.cfc :: test has _ prefix (_testRunner.cfc)
     [java]    [script] /home/runner/work/Lucee/Lucee/test/components/_Query.cfc :: test has _ prefix (_Query.cfc)
     [java]    [script] /home/runner/work/Lucee/Lucee/test/functions/_IsNull.cfc :: test has _ prefix (_IsNull.cfc)
     [java]    [script] /home/runner/work/Lucee/Lucee/test/functions/_SerializeJSON2.cfc :: test has _ prefix (_SerializeJSON2.cfc)
     [java]    [script] /home/runner/work/Lucee/Lucee/test/functions/_DeSerializeJSON.cfc :: test has _ prefix (_DeSerializeJSON.cfc)
     [java]    [script] /home/runner/work/Lucee/Lucee/test/datasource/_postgreSQL2.cfc :: test has _ prefix (_postgreSQL2.cfc)
     [java]    [script] /home/runner/work/Lucee/Lucee/test/a_debug_build/_A_error.cfc :: test has _ prefix (_A_error.cfc)
     [java]    [script] /home/runner/work/Lucee/Lucee/test/a_debug_build/_A_failure.cfc :: test has _ prefix (_A_failure.cfc)
     [java]    [script] /home/runner/work/Lucee/Lucee/test/a_debug_build/_MongoDB.cfc :: test has _ prefix (_MongoDB.cfc)
     [java]    [script] /home/runner/work/Lucee/Lucee/test/components/Administrator/TestArchive/Admin.cfc :: Not a test suite
     [java]    [script] /home/runner/work/Lucee/Lucee/test/functions/arrayFunctions.cfc :: test suite has skip=true
     [java]    [script] /home/runner/work/Lucee/Lucee/test/functions/CreateDynamicProxy/osgi/Completor.cfc :: Not a test suite
     [java]    [script] /home/runner/work/Lucee/Lucee/test/functions/custom/custom/Serialize.cfc :: Not a test suite
     [java]    [script] /home/runner/work/Lucee/Lucee/test/functions/CreateDynamicProxy/javaSetting/Test.cfc :: Not a test suite
     [java]    [script] /home/runner/work/Lucee/Lucee/test/functions/Duplicate/comps/some/B.cfc :: Not a test suite

For further documentation about this see Building Lucee 5 & 6 from source :: Lucee Documentation

Happy hacking!

1 Like