Efficient [re]deployment practices for a lucee war application

The conversation from a different topic prompts me to start over with a more generic use-case question. I have an older CF application that was extended to include other Java servlets. Since these servlets also needed a datasource, we were setting a JNDI resource in the configuration. From what I can tell, if the lucee.jar is in the classpath, and an application is deployed to a container that loads the LuceeServlet, the code will check for the dependent files in WEB-INF/lucee, and expand them if they don’t exist. (It does the same for the server files). Thus, these do not need to be included in a project source code. However, the Lucee web instance stores datasource info (and other configurations) in the WEB-INF/lucee/lucee-web.xml.cfm file, so this would need to get populated after the expansion, but before the application loads. @bdw429s suggested using CFConfig as part of the deployment process, which looks like it has some possibility, I’m just not sure the best way to incorporate it.

So, here’s my question: how can I include my configuration in my source code (for things like scheduled tasks), or externalized (for database settings), but have these loaded/injected into Lucee upon startup? The solution needs to assume that the Lucee instance files don’t exist initially. Does it make sense to write a servlet that loads after the Lucee core? Could I make use of the CFConfig services?

From my understanding Lucee will not overwrite things like XML configs if they already exist when Lucee first unpacks itself. You can set some app config in your Application.cfc, but not all of it, which is why I use CFConfig.

So, part of your build/deploy process could be

box cfconfig import from=mySettings.json to=/path/where/server/context/will/be toFormat=luceeServer
or...
box cfconfig import from=mySettings.json to=/path/where/web/context/will/be toFormat=luceeWeb

It’s your choice whether you decide to put the settings in the server or web context (I prefer server for single-context deploys). But if you write out Lucee configs to a directory that doesn’t yet exist (somethign CFConfig was built do to) then you’ll need to include the toFormat so CFConfig knows what type of Engine to use.

Then start your server as normal and when Lucee wakes up and expands itself into those directories, it should pick up the existing XML files and use them.

One challenge with that approach is that when the container extracts the war, it does so to a temp directory space that may not be completely predictable in each instance.

Another tact I was just looking at is including a WEB-INF/lucee/context/Web.cfc with an onWebStart method that could load the CFConfig services, read in a provided json file and write it to the (now) internally discoverable lucee-web.xml.cfm location.

I’m used to Tomcat where the war extracts to a folder named after the war. You can also use your web.xml to override where the Lucee server and web contexts are placed so they’re outside the war and in a predictable place on your drive.

I’m trying to keep this as self-reliant and container agnostic as possible. I’d like to be able to checkout my code base, and build and run locally, or deploy to an existing container.

I looked at trying to load the CFConfig services, but I see that it is dependent on numerous other packages; it’s not standalone enough to be used to configure the server on load.

There doesn’t appear to be a good process for bootstrapping a lucee instance after being loaded from a container.

I’m trying to keep this as self-reliant and container agnostic as possible

Then I think you need to move away from any scenario in which you deploy a war to a servlet container and then try and guess where on disk the servlet has expanded your war and then update files in that location. If you want to be self contained, build the war in the first place with the configuration present that you desire.

I’d like to be able to checkout my code base, and build and run locally, or deploy to an existing container.

I don’t see why you can’t do this though you’ll obviously need a build process to take the code and configuration for your site and wrap it up in a deployable WAR. Just to ask the question, is a WAR a hard requirement, or have you looked at just using CommandBox to run your sites. It uses Undertow as the servlet container, but it builds the war for you on the fly behind the scenes so all you need to have present is your web root and it starts up the code.

I looked at trying to load the CFConfig services, but I see that it is dependent on numerous other packages

It only requires that the CommandBox binary be present. Can you explain the “numerious” packages?

it’s not standalone enough to be used to configure the server on load.

I think that’s part of your problem. Why are you trying to do this on load? If you’re generating a war file, inject the configuration at that point in time. If you really want config to be a startup concern, just use CommandBox directly to run your servers. It has an interception model in place that is perfect for this and CFConfig already contains all the hooks for this to work out of the box.

There doesn’t appear to be a good process for bootstrapping a lucee instance after being loaded from a container.

That’s mostly because no one really does this. People building wars, build the config into the war in the first place. People automating deploys at run time either copy XML files around or use CFConfig.

I think it’s reasonable to expect a container to manage where it’s work files are. And, I think it’s also reasonable to expect an artifact not to have to be build for deployment to a specific environment (test artifact different from a production artifact).

This is why I’m trying to understand how to efficiently deploy an application with Lucee. It sounds like your suggesting that there isn’t, in fact, a way to deploy lucee applications without CommandBox.

I was referring to the statement that CFConfig is “an underlying service layer”, and that the usage section describes how to make programmatic calls. The code requires at least WireBox be present and configured for it to run; not only is it a named dependency, the process of injecting it uses WireBox to do so.

First reason is because I want to have an application artifact that is not built for a specific environment. Second reason is because the location that the configuration be injected (the lucee config) may not be present until load. (Lucee sees the web admin files are missing and creates them; this includes the config file) Yes, CFConfig is designed specifically to address it, and it would be a great tool. But I need to build it into my artifact, to be able to load on startup and apply the config.

It could be that no one really does this because the system doesn’t provide mechanisms to easily handle configuration at runtime. I’m just trying to leverage the functionality that is written into Lucee, namely the creation of necessary files if they are not present on load. It’s a nice feature and keeps my war file clear of boiler-plate code. I’m just struggling with the fact that it writes its own default config without a way to inject any additional info.

I think it’s also reasonable to expect an artifact not to have to be build for deployment to a specific environment

That’s a great goal, but it’s not always possible. For instance, I deploy tests to my test env, but not on production. I also apply different security locks downs on production. Many changes such as passwords or different DB settings are possible via JVM args or env vars however.

. It sounds like your suggesting that there isn’t, in fact, a way to deploy lucee applications without CommandBox.

That is not true, there are many ways to deploy Lucee without CommandBox. CommandBox is simply my favorite since it’s the most portable, most configurable, and most extensible. Full disclosure, I’m part of the CommandBox project. If you’re open to rethinking your deploys to a method that doesn’t involve war deploys on an existing servlet, I think it’s worth considering as we’ve already solved most all of these issues there.

I was referring to the statement that CFConfig is “an underlying service layer”

There are two CFConfig projects. One that is just the service layer, and another which is a CLI tool (a module for CommmandBox). The latter has no dependencies outside of the box binary being present on the machine. Once the module is installed, it generates commands you can run to manage config on disk.

First reason is because I want to have an application artifact that is not built for a specific environment. Second reason is because the location that the configuration be injected (the lucee config) may not be present until load. (Lucee sees the web admin files are missing and creates them; this includes the config file) Yes, CFConfig is designed specifically to address it, and it would be a great tool. But I need to build it into my artifact, to be able to load on startup and apply the config.

Right, so it’s hard to reply to that paragraph since I think there’s at least 3 different options being discussed here simultaneously and I’m not sure which comments of yours go with which approach.

  1. Pre build the war with all config so it’s ready to deploy
  2. Build a war and have a deploy-time process that modifies config on the fly
  3. Scrap the whole WAR idea and move to a server solution like CommandBox

And of course, other server ideas we’re not discussing include a “typical” installation of Lucee which embeds Tomcat, but I’m assuming you want something more self contained and ad-hoc than that.

Your issue with #1 seems to be that you don’t like having different war artifacts for different environments. That’s not necessarily true as there are tricks you can do inside Lucee with environment variables to make your config dynamic. But I would challenge your desire to have a single artifact. If it’s automated by a build process, who cares. In fact, there are benefits to having a fully warmed up artifact with the server pre-started, all config deployed, etc so when you drop it in production it’s ready to go, I would also add into the #1 category (even though it can apply to anything) that you can set quite a few settings in your Application.cfc. I assume you’re aware of this, but perhaps not. You can create basic things like datasource, mail servers, and cache connections via CFML, and even mix in env vars or Java sys props here. This is pretty powerful, but does not cover 100% of what you can set in the web admin UI which is why I use CFConfig (which covers maybe 95% of the web admin) Adobe’s Application.cfc support is pretty crap too which is another reason I don’t make a habit of using it.

Your issues with #2, I’m not sure, but they seem to be centered around the fact that Lucee creates the server and web context on first start and the conig files don’t exist until then. That is true, but all the more reason I think you should be warming up your wars so none of that needs to happen on deploy. At least for production. Nonetheless, you can script in config externally, but you are correct that you’d need to know where on the drive the war will go.

I’m not actually sure if you have any issues with #3 or not. Perhaps you have your heart set on Tomcat or whatever and that’s fine. CommandBox would replace your servlet entirely and is a CLI-driven approach to startup and and shut down servers in a container mindset where each site/war is a separate process/JVM. We’ve done a lot of work to make CommandBox servers turn-key, 100% portable, and driven off JSON files which is why I keep suggesting it. We’ve spent years solving these issues that the CF Engine itself doesn’t solve in CommandBox.

It could be that no one really does this because the system doesn’t provide mechanisms to easily handle configuration at runtime.

Yes, I think you have a valid point there. That is exactly why I created CF Config (which works for Lucee and Adobe all the same). Just to reiterate, Lucee DOES support:

  • A handful of one-off JVM props/env vars to toggle settings such as full null support,etc
  • A decent amount of programmatic configs in Application.cfc such as datasources, etc
  • Convenient access to the java system props and OS env vars in the server scope
  • A VERY SMALL set of features are available via the Admin API

For everything else, the people out there not using CFCOnfig are just manually copying XML files around. That’s really the only other solution and I think it sucks which is, again, why I wrote CFConfig.

I’m just trying to leverage the functionality that is written into Lucee, namely the creation of necessary files if they are not present on

Yep, it’s a noble goal, but IMO Lucee doesn’t give you enough out of the box.

It’s a nice feature and keeps my war file clear of boiler-plate code

I’m not sure what you mean there. A “warmed up” war is not just boilerplate, it’s a lightning fast deploy in production when seconds matter. At Ortus, we use Docker swarm so all of this uses CommandBox based docker image deploying to a swarm on Digital Ocean. I can click a button and have a new container up and serving traffic in seconds. You better believe we warm the heck out of our containers, but the Gitlab build server generates those automatically so it’s no skin off our teeth to have all the files in place and ready to go,

I’m just struggling with the fact that it writes its own default config without a way to inject any additional info.

Yep, I wish Lucee did more out of the box, but it doesn’t. Hence all my suggestions to you :slight_smile:

Just to confirm, since I can’t find a forum for any of the CommandBox tools, if I run cfconfig export to=path/myconfig.json from=path/lucee/, I should expect the json file to contain everything in the lucee config? Specifically, does it capture the debugger settings?

You can get support for CommandBox and CFConfig on our Google group
https://groups.google.com/a/ortussolutions.com/forum/#!forum/commandbox

on the Box Team Slack in the #commandbox channel:
https://boxteam.herokuapp.com/

On the public CFML Slack team in the #box-products channel
https://cfml-slack.herokuapp.com/

or by uttering the word “CommandBox” on Twitter :slight_smile:

To answer your question, yes it should pull all of the configuration. Note, if you don’t see everything you expected, you may be looking in the wrong context. There is a web and server context and CFConfig reads the separately depending on which one you’ve pointed to. For a war deploy, the distinction is fairly irrelevant and I make the habit of just sticking everything in the server context to simplify.

You might try configuring your application via environment variables and bypass the XML config files completely.

There are scant few settings that cannot be modified at start-up via ENV or system properties and the Application.cfc:

https://lucee.daemonite.io/t/lucee-configuration-options/321