After an update (complete replacement) from Lucee 5.3.3.62 to Lucee 5.5.3.7.47 we experience difficulties/errors when uploading files with german «Umlaute»
Malformed input or input contains unmappable characters: /pathtoupload/Öffnungszeiten.pdf
Our stack
CentOS Linux release 7.9.2009 (Core)
# locale
LANG=de_CH.UTF-8
LC_CTYPE="de_CH.UTF-8"
LC_NUMERIC="de_CH.UTF-8"
LC_TIME="de_CH.UTF-8"
LC_COLLATE="de_CH.UTF-8"
LC_MONETARY="de_CH.UTF-8"
LC_MESSAGES="de_CH.UTF-8"
LC_PAPER="de_CH.UTF-8"
LC_NAME="de_CH.UTF-8"
LC_ADDRESS="de_CH.UTF-8"
LC_TELEPHONE="de_CH.UTF-8"
LC_MEASUREMENT="de_CH.UTF-8"
LC_IDENTIFICATION="de_CH.UTF-8"
Servlet Container: Apache Tomcat/9.0.35|
Java:11.0.7 (AdoptOpenJDK) 64bit
OS: Linux (3.10.0-1160.24.1.el7.x86_64) 64bit
Architecture: 64bit
Server Settings Lucee
Language/Compiler > Template charset: UFT-8
Regional > Locale: English (United States)
Charset > Template charset: UFT-8
Charset > Web charset: UFT-8
Charset > Resource charset: UFT-8
Regarding these recommendations I have adjusted the start options. Unfortunately, this did not help.
setenv.sh > added -Dsun.jnu.encoding=UTF-8 -Dfile.encoding=UTF-8 to CATALINA_OPTS
# Tomcat memory settings
# -Xms<size> set initial Java heap size
# -Xmx<size> set maximum Java heap size
CATALINA_OPTS="-Xms6144m -Xmx6144m -XX:+UseG1GC -XX:MetaspaceSize=512m -XX:+ParallelRefProcEnabled -XX:MaxGCPauseMillis=200 -XX:InitiatingHeapOccupancyPercent=70 -XX:G1ReservePercent=15 -XX:ParallelGCThreads=20 -XX:ConcGCThreads=5 -XX:+AggressiveOpts -Dsun.jnu.encoding=UTF-8 -Dfile.encoding=UTF-8";
# additional JVM arguments can be added to the above line as needed, such as
# custom Garbage Collection arguments.
export CATALINA_OPTS;
I am surprised that others have not already experienced this problem. At least I have not found anything about it. Any ideas, suggestions or tips to fix this problem are highly appreciated.
I have found a short term solution for my applications:
If you set the name of the file when saving via cffile, it works …
<cfscript>
isDefined("Form.FileContents") {
filename = GetPageContext().formScope().getUploadResource("FileContents").getName();
renamed_filename = reReplace(filename, '[^a-zA-Z0-9\.]', '', 'all');
FileUpload( destination='/uploadpath/' & renamed_filename, fileField=FileContents, accept='application/pdf', nameConflict="overwrite", mode="644" );
}
</cfscript>
but if you do not name the file explicitly, the upload fails
Catch [Entries: 12]
additional Struct (ordered)
Detail string
ErrorCode string 0
Extended_Info string
ExtendedInfo string
Index number -1
Input string /upload_path/Öffnungszeiten.pdf
Message string Malformed input or input contains unmappable characters: /upload_path/Öffnungszeiten.pdf
Reason string Malformed input or input contains unmappable characters
StackTrace string lucee.runtime.exp.NativeException: Malformed input or input contains unmappable characters: /upload_path/Öffnungszeiten.pdf
at java.base/sun.nio.fs.UnixPath.encode(UnixPath.java:145)
at java.base/sun.nio.fs.UnixPath.(UnixPath.java:69)
at java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
at java.base/java.io.File.toPath(File.java:2292)
at lucee.commons.io.res.type.file.FileResource.copyTo(FileResource.java:102)
at lucee.commons.io.IOUtil.copy(IOUtil.java:126)
at lucee.runtime.tag.FileTag._actionUpload(FileTag.java:988)
at lucee.runtime.tag.FileTag.actionUpload(FileTag.java:831)
at lucee.runtime.tag.FileTag.doStartTag(FileTag.java:412)
at _mailtest.upload_cfm$cf$bj.call(/_mailtest/upload.cfm:13)
at lucee.runtime.PageContextImpl._doInclude(PageContextImpl.java:945)
at lucee.runtime.PageContextImpl._doInclude(PageContextImpl.java:837)
at lucee.runtime.listener.ModernAppListener._onRequest(ModernAppListener.java:216)
at lucee.runtime.listener.MixedAppListener.onRequest(MixedAppListener.java:42)
at lucee.runtime.PageContextImpl.execute(PageContextImpl.java:2416)
at lucee.runtime.PageContextImpl._execute(PageContextImpl.java:2406)
at lucee.runtime.PageContextImpl.executeCFML(PageContextImpl.java:2381)
at lucee.runtime.engine.Request.exe(Request.java:43)
at lucee.runtime.engine.CFMLEngineImpl._service(CFMLEngineImpl.java:1170)
at lucee.runtime.engine.CFMLEngineImpl.serviceCFML(CFMLEngineImpl.java:1116)
at lucee.loader.engine.CFMLEngineWrapper.serviceCFML(CFMLEngineWrapper.java:97)
at lucee.loader.servlet.CFMLServlet.service(CFMLServlet.java:51)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:747)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:373)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1590)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: /upload_path/Öffnungszeiten.pdf
... 45 more
We have something like this.
It renames the file with date, time & username, description (drop down), session id and or what the file was suppose to be for.
The result is files outside of webroot are named like this:
04-22-2021-1301045-johnsmith-HRJOBOPENINGXYZ-SITE4-BCADEFGHLBCDAEFFAE84-resume-isPDF
can you file a bug in jira?