Duplicating a Closure

Hi all,

First post on the forum. I’m trying to implement a helper cfc something a bit like an Elixir Agent to help people cope with locking:

a = new Agent({name: "Robin", limbs: 4});
delta = -1;
a.update(function(person) {person.limbs += delta});
value1 = a.updateAndGet(function(person) {person.limbs += delta}); 
value2 = a.get();

I’m using duplicate on the init parameter and return values to give users thread-safety. The only thing I can’t lock down is by-reference variables being used in the lambda.

I notice that in in Closure.java line 68 in the duplicate method there’s a TODO saying

duplicate variables as well?

My (very likely incorrect) understanding is that variables holds the original PageContext of where the closure was created. I doubt we’d want to force pass-by-value of the closure, but for my purposes it would be lovely to have a separate function to which I could pass my function and have it duplicate the variables for me, so that they would be thread-safe when accessed inside the function.

I know that trying to port ideas from languages with immutable variables is always going to run into some issues like this, but if there’s anything I care about in the future of CFML/Lucee it’s great support for concurrent and parallel programming - Agents are one abstraction that can help.

Cheers,
Robin

2 Likes

wild guess - try
var delta = -1;

Hi AJ,

The problem is that as the writer of the Agent cfc I can’t control whether or not the user is var-ing their variables - I’d just be replacing “remember to always cflock” with “remember to var your variables”. I’m looking for a way to disconnect the closure passed to the Agent instance’s methods from the scope it was declared in by giving it a duplicate of that scope.

Cheers,
Robin

Here’s an example of the problem:

person = new Agent({name: "Robin", 
                   limbs: [
                       {name: "left leg"}, 
                       {name: "right leg"}, 
                       {name: "left arm"}
                    ]});

// the contents of my agent should only be able to be
// changed in a thread-safe manner, for example:

person.update(function(person) {person.limbs.append({name: "right arm"})});

// but if I accidentally use something from the closure
// that is pass-by-reference this will not be true

extraLimb = {name: "tail"};
person.update(function(person) {person.limbs.append(extraLimb)});

// This should not update the contents of the Agent, but it does
extraLimb.name = "tentacle";

dump(person.get());

And here’s the current source of Agent.cfc. The final version might use a separate thread to do update(), but otherwise it wouldn’t need to be much more complicated than this. I would use the proposed duplicateClosure() function inside each method on the fn argument before calling it.

 component {

      LOCK_TIMEOUT = 1

      value = ""
      lockName = ""

      public function init(initValue) {
         value = duplicate(initValue);
         lockName = createUUID();
         return this;
      }

      public function updateAndGet(fn) {
         var getValue = 0;
  
         // duplicateClosure(fn);

         lock name=#lockName# type="exclusive" timeout=LOCK_TIMEOUT {
            fn(value);
            getValue = duplicate(value);
         }
  
         return getValue;  
      }

      public function update(fn) {

         // duplicateClosure(fn);

         lock name=#lockName# type="exclusive" timeout=LOCK_TIMEOUT {
            fn(value);
         }
      }

      public function get(fn = identity) {
         var getValue = 0;

         // duplicateClosure(fn);
  
         lock name=#lockName# type="readonly" timeout=LOCK_TIMEOUT {
            getValue = duplicate(value);
         }
  
         return fn(getValue);
      }

      private function identity(x) {
         return x;
      }
    }
// This should not update the contents of the Agent, but it does
extraLimb.name = "tentacle";

I don’t see anything wrong with an update to extraLimb being reflected in your person object. That might not be desirable to you, but structs in CFML are passed by reference so it makes sense. This doesn’t seem to have anything at all do to with closures or thread safety, and everything to do with the fact that you’re passing a struct by reference and then updating it.

Also, what are you doing with the cflock? I don’t understand what you’re trying to accomplish there. That would only seem necessary if your person object was being access by multiple threads, AND at some point during the update process was in an inconsistent state. And in that instance, you would wrap the part of the update that needed synchronizing with an exclusive lock named specific to that INSTANCE and a read only lock on the accessor.

And finally, can you explain what you expect to happen when duplicating a closure? A closure is not a data structure and has no state. It’s just a reference to executable code. I don’t understand how or why you would duplicate that.

Hi BDW,

Yes, the only point of an Agent is thread safety in concurrent programming. Locking access to shared variables has always seemed to spook CF developers - how do you ensure that access to a shared variable is correctly locked every time? The Agent component uses object encapsulation to enforce the correct use of locks. Rather than put a resource directly in a shared scope we put an agent wrapping the resource in the shared scope and access it via the agent methods. It also allows a developer to easily and transparently use uniquely named locks, which seem to be rarely used in the CFML code I’ve seen over the years.

A closure is all about state - the variables scope where I wrote person.update(function(person)) is available to the body of that function until it is garbage collected. There is no way in CFML once a closure is created to disconnect the function from that scope, and so, despite everything else I cannot guarantee the user of Agent that the state it is protecting will be thread safe. As it is, Agent can only protect simple values. If I had a way to duplicate the variables state and make the duplicate available to the body of the function this would extend the protection to structs and object instances. This is what the proposed duplicateClosure(function) function would do.

Cheers,
Robin

Come to think of it, we could just augment the behaviour of the existing duplicate() function when a closure is passed to it.

1 Like