Errors upgrading to 5.3.9

[gooutside]

I installed this update and had a near total meltdown. Upon install, I received a blue and white status 500 error page stating that “datasource [name] does not exist.” I could not even access the admin pages because of this error.

To resolve it, I commented out the datasource line in /opt/lucee/config/server/lucee-server/context/lucee-server.xml. Then I was able to open the administrator pages, but I was asked to create a new password. When I did so, it was as though I’d started an all-new server from scratch; no configuration was saved.

During this time, the server would not load any other CFM pages; they all returned blank pages.

I used my access to the administrator to DOWNGRADE back, then moved a backup copy of lucee-server.xml to replace the stupid new one.

Suffice to say there will be no upgrading for me. Total disaster, top to bottom.

[andreas]

In former times (over 10 years ago) everything was saved in the Administrators server or web context. If you wipe it out, you loose the settings. But that has changed because of exactly that issue you had and for portablity. The vast number of these settings have been migrated to be set within your Application.cfc. You should add all these configurations like datasources, mailservers, mappings, charsets, session configs and everything else to your Application.cfc. The Lucee Administrator also offers a wonderful export tool to copy all these settings to your Application.cfc.

If you can’t use it, you should use tools for backing up these settings, such as CommandBox with cfconfig.

Also, make sure to move and create your scheduler tasks tag based with cfschedule.

[micstriit]

@gooutside I’m sorry you had this troubles. That is really strange.
When Lucee cannot load the XML file because of a syntax error in the file itself, it renames the file to /opt/lucee/config/server/lucee-server/context/lucee-server.xml.1.buggy and creates a new blank version of that file.

Lucee never deletes or strips that file, you write:

@gooutside

To resolve it, I commented out the datasource line in /opt/lucee/config/server/lucee-server/context/lucee-server.xml. Then I was able to open the administrator pages, but I was asked to create a new password. When I did so, it was as though I’d started an all-new server from scratch; no configuration was saved.

So it seem you did not get a blank version of the file, so what did actually change?
was maybe the lucee-web.xml.cfm reset?
Can you please give me that “datasource” line that caused the issue, of course with sensitive data altered.

[gooutside]

Good afternoon,

I’ve attempted to do this again by cloning my instance and trying out a few different things.

First, I tried a straight up repeat of the steps and once again got the same error/issue.

Second, I started from scratch with a cloned instance again, but this time:

• Copied the datasource connection into the Application.cfc file.
• Then I removed it from the datasource from the web admin datasources page.
• Then I restarted the server and confirmed that the Application.cfc datasource connection was successful. It was.
• I then used the web admin to update Lucee to 5.3.9.133.

The same error returned, unabated. It should be noted this is a very different error page than the typical “orange and marigold” colored errors that Lucee returns. This one is black text on a white background with a blue header bar. Picture is below:

I’m not really sure what else to do other than attempt a reconfiguration of the entire server from scratch, and to be clear, I do not want to do this. Additionally, there was no “lucee-server.1.buggy” created this time.

Any ideas on what could be causing this?

[bdw429s]

The reason you’re seeing Tomcat’s default error is because Lucee is blowing up at a very early point and the exception is flowing all the way up to the servlet. It appears you’re using some custom logging config to send your log messages to a database. To be clear, is this “wx” datasource that the logging is using the same one that you moved to Application.cfc? If so, I wouldn’t expect the DB logger to be able to find it.

The fact that you’re using a custom DB logger is rather important information that I didn’t see you mention before (when reporting an error, please always include the full stack trace!), and the people above who suggested you move your datasource definitions to the Application.cfc, probably didn’t realize they were being used in that manner. I would try putting the datasource back into the admin, and if necessary, you may need to delete your custom logger config from the XML file, if the admin UI itself is being affected by the logger error.

[bdw429s]

@gooutside I’ve broken your thread off to a new topic so we can better help you without spamming the original thread.

[gooutside]

This was the issue. To confirm it, I commented out the relevant <logger ... sections from the lucee-server.xml file. Upon a restart, everything went back to normal.

I configured the DB logger through the stock server admin web portal; this was not some kind of special thing I invented to serve my own needs. It would appear that this is some kind of breaking issue when attempting an upgrade, perhaps something for the dev team to consider.

I will be attempting another from-scratch upgrade, tomorrow, in which I will remove the DB logging entries from the “Logging” settings in the server administrator pages, upgrading, and then adding them back to see if they still work.

If I can supply any additional information or resources to help with this issue, please let me know.

[bdw429s]

@gooutside

this was not some kind of special thing I invented to serve my own needs.

Right, but it isn’t a “stock” configuration and to be honest, you’re one of the only people I’ve ever run across who use it. Nothing wrong with using it, but you have to include these sort of details when you report an error, especially if it’s details nobody would assume.

@gooutside

If I can supply any additional information or resources to help with this issue

I guess I’m a little confused what the issue even is at this point. The only stack trace you’ve posted in this entire thread was because you were trying to use a datasource in a logger but the DSN wasn’t defined in the admin UI. In your last message you confirmed putting the datasource in the correct place works. So, what exactly isn’t working at this point? How do you reproduce the issue? What error do you see when the issue occurs?

[gooutside]

Good morning. I apologize if I was not clear. The issue was, indeed, related to sending logs to a datasource.

When attempting the upgrade, the datasource did, in fact, exist. I was able to successfully upgrade without errors by using the following workaround:

1. Create a clone of existing server
2. Enter web admin and delete logging items (in Settings >> Logging) that sent logs to datasource
3. Upgrade through web admin
4. Upgrade successful
5. Re-add logging-to-datasource items through web admin (Settings >> Logging)
6. Logs resume appending to datasource

I do not know whether this is an issue directly related to something in my datasource definition or whether this is due to an issue with the logging-to-datasource option breaking the upgrade.

But, to be clear, removing the logging-to-datasource options is the key step that allows the upgrade to succeed.

[bdw429s]

This info is good, but there’s still a lot missing for anyone to even try and duplicate this.

• Where was the datasource logger defined? (web or sever context)
• Where was the datasource itself defined? (web or server context)
• What kind of datasource is this? (MySQL, SQL Server, Postgres, etc)
• If you attempt to upgrade without your workaround, what exactly happens? An error? Something else?

There are more questions, but I can’t even ask them until we get the questions above nailed down.

[gooutside]

@bdw429s

• Where was the datasource logger defined? (web or sever context)

It was defined in the server context.

@bdw429s

• Where was the datasource itself defined? (web or server context)

Also the server context.

@bdw429s

• What kind of datasource is this? (MySQL, SQL Server, Postgres, etc)

MySQL 5.7

@bdw429s

• If you attempt to upgrade without your workaround, what exactly happens? An error? Something else?

Upon upgrade (without the workaround), I get the Tomcat error, as seen in the photo in the earlier post above.

I am happy to answer additional questions to help diagnose the issue, or perform different steps if needed.

[bdw429s]

@gooutside

Upon upgrade (without the workaround), I get the Tomcat error, as seen in the photo in the earlier post above.

Ok, this is great information. I wasn’t clear above if you only received the error about the datasource missing AFTER you had moved its definition to the Application.cfc since you only reported that stack, trace after making that change.

• Just to confirm, in that case, are you able to see if the datasourse appears in the admin UI, or does the Tomcat error prevent you from even viewing the admin?
• Can you confirm that the datasource is still defined in the server XML config file? I’m unclear if you had the file wiped just one one of the attempts, of the entire XML file is going away every time.
• If the entire XML file isn’t being wiped any longer, can you do a compare before and after your workaround to see the difference between the working and non-working version? If the XML file is getting wiped every time, can you provide a copy of it via DM to Micha as he requested from you above so he can reproduce?

Now that I have a clearer understanding, if I get a few minutes, I’ll try and reproduce it on my own as well.

[Zackster]

I agree, there’s definitely a baby and bath water bug here, I’ve encountered it myself

I even know there’s a bug in Jira somewhere I filed about this problem, alas I couldn’t find it, filtering in Lucee Jira by author:zac doesn’t really narrow things down anymore!!