Cfdirectory seems substantially faster than directoryList, why?

dev support
1

1st discovery: cfdirectory (or directory inside script) runs 20x faster than directoryList … what gives?
Example code below.

2nd observation: I cannot use the result with .each( reading each file and setting parallel to true ) to thread these quickly. Parallel actually slows processing down vs serial.
Are file READ operations always locked (cannot be threaded) or am I bumping into a possible environment issue?

<cfscript>
loop times=10 {


	variables.startTick = getTickCount();
	directory name="variables.files" filter="*.cfm" action="list" directory="c:/inetpub/wwwroot" listinfo="name" ;
	dump(var="cfdirectory took #getTickCount()-variables.startTick#ms", format="text");
	// cfdirectory takes ~20ms



	variables.startTick = getTickCount();
	directoryList(
		path = "c:/inetpub/wwwroot",
		listinfo = "name",
		filter = "*.cfm",
		type = "file"
	)
	dump(var="directoryList took #getTickCount()-variables.startTick#ms", format="text");
	// directoryList takes ~400ms

}
</cfscript>
2

TLDR; “parallel processing is not suited to sets of tasks each of which is quick to execute” where “quick”<= 5ms or so.

If the files you are reading are small then it’s quite possible the read operation is sub 5ms.

3 Likes
3

Thanks. Yes, threads add slight overhead. My expectation with several hundred files was some speed improvement and I’m not seeing any.

  • Will try with larger files to satisfy my curiosity

It was the ~400ms that was a bigger issue here. Refactoring the code to use cfdirectory helped a lot.

4

In further tests and trying various versions of 5.3.8.x / 5.3.9.x I’m still seeing a measurable difference. The absolute of 20x may have been a poor title as the difference depends on many factors.

  • Can anyone else confirm this performance bug?
5

Bump

6

If you’re asking why it’s slower, use the FusionReactor profiler to identify what is running the most inside the thread.

1 Like
7

Will do and report back. Thanks

For our initiate needs we switched to the tag as the time it took using the function was 20x slower.

8

Wow, if I were to bet, it’s related to how much data that function/tag gets about each file by default. There are a handful of tickets in Lucee’s backlog about this sort of thing. It’s one of the things that makes working with large dirs in CFML slower than Java because Lucee loops over every file to get details on then to build up its list, even if you never use that data.