Query Caching

dev google-group-archive
1

In looking at the CF 2016 version I noticed that cfquery has the ability to
specify a cacheid in there query tag. This allows you to pass an array of
id’s for clearing that cache by those id’s. I created my own udf for lucee
using mongodb as my query caching with this functionality but would prefer
to no longer support my own udf and use the built in functionality. Is
there a plan to bring lucee cfquery up to compatibility with CF? I believe
this cacheid has been in the code for a year or two.

https://helpx.adobe.com/coldfusion/cfml-reference/coldfusion-tags/tags-p-q/cfquery.html

2

ANYONE??? Is anyone out there?On Monday, June 27, 2016 at 9:30:32 PM UTC-7, Lyle Karstensen wrote:

In looking at the CF 2016 version I noticed that cfquery has the ability
to specify a cacheid in there query tag. This allows you to pass an array
of id’s for clearing that cache by those id’s. I created my own udf for
lucee using mongodb as my query caching with this functionality but would
prefer to no longer support my own udf and use the built in functionality.
Is there a plan to bring lucee cfquery up to compatibility with CF? I
believe this cacheid has been in the code for a year or two.

https://helpx.adobe.com/coldfusion/cfml-reference/coldfusion-tags/tags-p-q/cfquery.html

3

Lyle,

For this sort of thing, best to submit an enhancement request.

That said, a few days ago I was experimenting with using query caching in
ACF 11 and found it much too buggy to rely on. I was trying to set the
cacheId to be able to programmatically clear the cache, but simply couldn’t
get it to work. I wound up implementing it by using cachePut() and
cacheGet(), which worked perfectly from the get go. Here’s some code to
explain:

var qOpenWorkPeriods = cacheGet(
“sal-findOpenPayableWorkPeriodHoursInDateRange”);
if ( isNull( qOpenWorkPeriods ) ) {
var qOpenWorkPeriods = findOpenPayableWorkPeriodHoursInDateRange(
arguments.companyId,
arguments.startDate, arguments.endDate );
cachePut(“sal-findOpenPayableWorkPeriodHoursInDateRange”, qOpenWorkPeriods,
createTimespan(0,0,0,30));
}

To me, it is much more straightforward to use the cache functions for
caching, and they work. You’ll find these cache methods also implemented in
Lucee, and I’d suggest doing it this way. In both ACF and Lucee,
CacheRemove() will take either a list or array of id’s.

In ACF, the cacheId’s for queries are buried in an automatic region or
something, and no matter what I tried, setting a custom region, using dot
notation to try and drill down to the cacheId, using the cacheId without
the region, hunting for them using getAllCacheIds, cursing at the screen
… nothing worked. I may simply be an idiot missing something obvious, but
I will never try to use a cacheId on a query again in ACF.

Aria Media Sagl
+41 (0)76 303 4477 cell
skype: ariamediaOn Sat, Jul 9, 2016 at 4:57 PM, Lyle Karstensen <@Lyle_Karstensen> wrote:

ANYONE??? Is anyone out there?

On Monday, June 27, 2016 at 9:30:32 PM UTC-7, Lyle Karstensen wrote:

In looking at the CF 2016 version I noticed that cfquery has the ability
to specify a cacheid in there query tag. This allows you to pass an array
of id’s for clearing that cache by those id’s. I created my own udf for
lucee using mongodb as my query caching with this functionality but would
prefer to no longer support my own udf and use the built in functionality.
Is there a plan to bring lucee cfquery up to compatibility with CF? I
believe this cacheid has been in the code for a year or two.

https://helpx.adobe.com/coldfusion/cfml-reference/coldfusion-tags/tags-p-q/cfquery.html

–
Win a ticket to dev.objective from Lucee via Twitter, see
http://bit.ly/1UbTMWj for details, good luck and see you there…

You received this message because you are subscribed to the Google Groups
“Lucee” group.
To unsubscribe from this group and stop receiving emails from it, send an
email to lucee+unsubscribe@googlegroups.com.
To post to this group, send email to lucee@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/lucee/936e77c3-6860-4faf-98f0-455716c8773a%40googlegroups.com
https://groups.google.com/d/msgid/lucee/936e77c3-6860-4faf-98f0-455716c8773a%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

4

Hi Lyle,

If this feature is missing from Lucee please check the Lucee JIRA (
https://luceeserver.atlassian.net/) and see if there is a ticket for it, if
not please add one and then it will at least be there to be looked at at
some point in the future.

Kind regards,

Andrew
about.me http://about.me/andrew_dixon - mso http://www.mso.net - Lucee
Association Member http://lucee.orgOn 9 July 2016 at 18:56, Lyle Karstensen <@Lyle_Karstensen> wrote:

Thanks for the reply, that is good to know. I guess I will stick with what
I already built.

FYI

I have implemented a Built In Tag that has been doing my query caching for
years (since moving to railo). I have a query tag called cfiqquery that is
a direct replacement for cfquery. I added 3 parameters (action, tags and
compress). I also built my own cfiqqueryparam tag since you cannot use
cfqeryparam outside the cfquery tag. All of this code implements query and
queryparam with a layer of caching built on mongodb.

If you want to clear cache that you have tagged you simply call

My query caching is so fast that a single mongo db replica set can easily
handle 2000 requests per second. We actually recently had an issue where
the data coming back from cache was too big and was maxing out our 1g
network. Therefore I implemented gzip on the json representation of the
query to stop that problem. Currently our caching layer responds in roughly
2ms and went from a constant 100MBps to about 12MBps.

–
Win a ticket to dev.objective from Lucee via Twitter, see
http://bit.ly/1UbTMWj for details, good luck and see you there…

You received this message because you are subscribed to the Google Groups
“Lucee” group.
To unsubscribe from this group and stop receiving emails from it, send an
email to lucee+unsubscribe@googlegroups.com.
To post to this group, send email to lucee@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/lucee/0f2fa47f-1a36-4682-ac8f-5e782d161da2%40googlegroups.com
https://groups.google.com/d/msgid/lucee/0f2fa47f-1a36-4682-ac8f-5e782d161da2%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

5

My point regarding caching is that the language interface should be kept
simple, rather than going down the route of having separate ways to cache
different data types. There is a tendency from the ACF devs to keep
throwing “features” in the language without much if any thought to design -
I think someone called it PHP-ification, where you wind up with a bunch of
different ways to do something. It becomes a mess, more difficult to learn
than necessary, that isn’t well maintained, and difficult to develop and
maintain programming tools for. The way this cacheID attribute has been
implemented in ACF is a case in point. To me, the solution is to center on
one, well designed and well developed way to cache stuff, resisting this
tendency to keep throwing multiple ways to do things into CFML, as quickly
as possible, releasing them full of bugs, and then moving on to the next
release cycle to do the same.

Aria Media Sagl
+41 (0)76 303 4477 cell
skype: ariamediaOn Mon, Jul 11, 2016 at 2:30 AM, Lyle Karstensen <@Lyle_Karstensen> wrote:

I think you may misunderstand how the caching works. The cache storage is
the same for each type and has attributes stored in some sort of structure.
If you look at the cache and how it is stored it is something like

Array
{
id:somecacheid,
data:[‘blah1’,‘blah2’],
expires:somedate,
hits:(number of times matched)
cacheid:‘blahblabh’
}

String
{
id:hash(),
data:‘some string data’,
expires:somedate,
hits:(number of times matched)
cacheid:‘blahblabh’
}

Struct

{
id:hash(),
data:{key:‘value’,key:‘value’},
expires:somedate,
hits:(number of times matched)
cacheid:‘blahblabh’
}

When you do a cached query it creates a json base64 representation of the
query then reverses that.

On Saturday, July 9, 2016 at 9:35:27 AM UTC-7, Nando Breiter wrote:

by the way … the main reason I don’t like the cacheId approach on a
query is that there are other data types that may need to be cached. Taking
this approach, it seems we supposedly need the same cacheID attribute for
lists, arrays, structs and components. Is there a cfarray tag? Nope. Where
to put this cacheID attribute for other datatypes? This line of reasoning
seems to imply we would then also need tags for lists, arrays and structs,
<cfarray … > <cflist … > <cfstruct … > so we had a place to hang this
cacheID attribute, and that gets horribly awkward.

Cache functions for each datatype that would take a cacheId argument is
another possibility, ArrayCachePut(), ArrayCacheGet(), but unnecessary. A
single set of cache functions for all data types makes much more
sense.

So yes, someone is out here, and he thinks using CachePut(), CacheGet()
and CacheRemove() is the right way to handle caching in CFML.

Aria Media Sagl
+41 (0)76 303 4477 cell
skype: ariamedia

On Sat, Jul 9, 2016 at 5:39 PM, Nando Breiter na...@aria-media.com wrote:

Lyle,

For this sort of thing, best to submit an enhancement request.

That said, a few days ago I was experimenting with using query caching
in ACF 11 and found it much too buggy to rely on. I was trying to set
the cacheId to be able to programmatically clear the cache, but simply
couldn’t get it to work. I wound up implementing it by using cachePut()
and cacheGet(), which worked perfectly from the get go. Here’s some code to
explain:

var qOpenWorkPeriods = cacheGet(
“sal-findOpenPayableWorkPeriodHoursInDateRange”);
if ( isNull( qOpenWorkPeriods ) ) {
var qOpenWorkPeriods = findOpenPayableWorkPeriodHoursInDateRange(
arguments.companyId, arguments.startDate, arguments.endDate );
cachePut(“sal-findOpenPayableWorkPeriodHoursInDateRange”,
qOpenWorkPeriods, createTimespan(0,0,0,30));
}

To me, it is much more straightforward to use the cache functions for
caching, and they work. You’ll find these cache methods also implemented in
Lucee, and I’d suggest doing it this way. In both ACF and Lucee,
CacheRemove() will take either a list or array of id’s.

In ACF, the cacheId’s for queries are buried in an automatic region or
something, and no matter what I tried, setting a custom region, using dot
notation to try and drill down to the cacheId, using the cacheId without
the region, hunting for them using getAllCacheIds, cursing at the screen
… nothing worked. I may simply be an idiot missing something obvious, but
I will never try to use a cacheId on a query again in ACF.

Aria Media Sagl
+41 (0)76 303 4477 cell
skype: ariamedia

On Sat, Jul 9, 2016 at 4:57 PM, Lyle Karstensen lylekar...@gmail.com wrote:

ANYONE??? Is anyone out there?

On Monday, June 27, 2016 at 9:30:32 PM UTC-7, Lyle Karstensen wrote:

In looking at the CF 2016 version I noticed that cfquery has the
ability to specify a cacheid in there query tag. This allows you to pass an
array of id’s for clearing that cache by those id’s. I created my own udf
for lucee using mongodb as my query caching with this functionality but
would prefer to no longer support my own udf and use the built in
functionality. Is there a plan to bring lucee cfquery up to compatibility
with CF? I believe this cacheid has been in the code for a year or two.

https://helpx.adobe.com/coldfusion/cfml-reference/coldfusion-tags/tags-p-q/cfquery.html

–
Win a ticket to dev.objective from Lucee via Twitter, see
http://bit.ly/1UbTMWj for details, good luck and see you there…

You received this message because you are subscribed to the Google
Groups “Lucee” group.
To unsubscribe from this group and stop receiving emails from it, send
an email to lucee+un...@googlegroups.com.
To post to this group, send email to lu...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/lucee/936e77c3-6860-4faf-98f0-455716c8773a%40googlegroups.com
https://groups.google.com/d/msgid/lucee/936e77c3-6860-4faf-98f0-455716c8773a%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

–
Win a ticket to dev.objective from Lucee via Twitter, see
http://bit.ly/1UbTMWj for details, good luck and see you there…

You received this message because you are subscribed to the Google Groups
“Lucee” group.
To unsubscribe from this group and stop receiving emails from it, send an
email to lucee+unsubscribe@googlegroups.com.
To post to this group, send email to lucee@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/lucee/48deef79-278b-42d7-a0e3-8635a1ebd33e%40googlegroups.com
https://groups.google.com/d/msgid/lucee/48deef79-278b-42d7-a0e3-8635a1ebd33e%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

6

I think you may misunderstand how the caching works. The cache storage is
the same for each type and has attributes stored in some sort of structure.
If you look at the cache and how it is stored it is something like

Array
{
id:somecacheid,
data:[‘blah1’,‘blah2’],
expires:somedate,
hits:(number of times matched)
cacheid:‘blahblabh’
}

String
{
id:hash(),
data:‘some string data’,
expires:somedate,
hits:(number of times matched)
cacheid:‘blahblabh’
}

Struct

{
id:hash(),
data:{key:‘value’,key:‘value’},
expires:somedate,
hits:(number of times matched)
cacheid:‘blahblabh’
}

When you do a cached query it creates a json base64 representation of the
query then reverses that.On Saturday, July 9, 2016 at 9:35:27 AM UTC-7, Nando Breiter wrote:

by the way … the main reason I don’t like the cacheId approach on a
query is that there are other data types that may need to be cached. Taking
this approach, it seems we supposedly need the same cacheID attribute for
lists, arrays, structs and components. Is there a cfarray tag? Nope. Where
to put this cacheID attribute for other datatypes? This line of reasoning
seems to imply we would then also need tags for lists, arrays and structs,
<cfarray … > <cflist … > <cfstruct … > so we had a place to hang this
cacheID attribute, and that gets horribly awkward.

Cache functions for each datatype that would take a cacheId argument is
another possibility, ArrayCachePut(), ArrayCacheGet(), but unnecessary. A
single set of cache functions for all data types makes much more sense.

So yes, someone is out here, and he thinks using CachePut(), CacheGet()
and CacheRemove() is the right way to handle caching in CFML.

Aria Media Sagl
+41 (0)76 303 4477 cell
skype: ariamedia

On Sat, Jul 9, 2016 at 5:39 PM, Nando Breiter <na...@aria-media.com <javascript:>> wrote:

Lyle,

For this sort of thing, best to submit an enhancement request.

That said, a few days ago I was experimenting with using query caching in
ACF 11 and found it much too buggy to rely on. I was trying to set the
cacheId to be able to programmatically clear the cache, but simply couldn’t
get it to work. I wound up implementing it by using cachePut() and
cacheGet(), which worked perfectly from the get go. Here’s some code to
explain:

var qOpenWorkPeriods = cacheGet(
“sal-findOpenPayableWorkPeriodHoursInDateRange”);
if ( isNull( qOpenWorkPeriods ) ) {
var qOpenWorkPeriods = findOpenPayableWorkPeriodHoursInDateRange(
arguments.companyId, arguments.startDate, arguments.endDate );
cachePut(“sal-findOpenPayableWorkPeriodHoursInDateRange”,
qOpenWorkPeriods, createTimespan(0,0,0,30));
}

To me, it is much more straightforward to use the cache functions for
caching, and they work. You’ll find these cache methods also implemented in
Lucee, and I’d suggest doing it this way. In both ACF and Lucee,
CacheRemove() will take either a list or array of id’s.

In ACF, the cacheId’s for queries are buried in an automatic region or
something, and no matter what I tried, setting a custom region, using dot
notation to try and drill down to the cacheId, using the cacheId without
the region, hunting for them using getAllCacheIds, cursing at the screen
… nothing worked. I may simply be an idiot missing something obvious, but
I will never try to use a cacheId on a query again in ACF.

Aria Media Sagl
+41 (0)76 303 4477 cell
skype: ariamedia

On Sat, Jul 9, 2016 at 4:57 PM, Lyle Karstensen <lylekar...@gmail.com <javascript:>> wrote:

ANYONE??? Is anyone out there?

On Monday, June 27, 2016 at 9:30:32 PM UTC-7, Lyle Karstensen wrote:

In looking at the CF 2016 version I noticed that cfquery has the
ability to specify a cacheid in there query tag. This allows you to pass an
array of id’s for clearing that cache by those id’s. I created my own udf
for lucee using mongodb as my query caching with this functionality but
would prefer to no longer support my own udf and use the built in
functionality. Is there a plan to bring lucee cfquery up to compatibility
with CF? I believe this cacheid has been in the code for a year or two.

https://helpx.adobe.com/coldfusion/cfml-reference/coldfusion-tags/tags-p-q/cfquery.html

–
Win a ticket to dev.objective from Lucee via Twitter, see
http://bit.ly/1UbTMWj for details, good luck and see you there…

You received this message because you are subscribed to the Google
Groups “Lucee” group.
To unsubscribe from this group and stop receiving emails from it, send
an email to lucee+un...@googlegroups.com <javascript:>.
To post to this group, send email to lu...@googlegroups.com
<javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/lucee/936e77c3-6860-4faf-98f0-455716c8773a%40googlegroups.com
https://groups.google.com/d/msgid/lucee/936e77c3-6860-4faf-98f0-455716c8773a%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.